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ABSTRACT 

Information was gathered about how Technical and 
Further Education (TAPE) agencies throughout Australia were 
implementing and using performance indicators. The context of TAFE^s 
perceived mission and corporate goals were considered. A history of 
the development of national performance indicators in TAPE was 
compiled. The project drew heavily on letters and accompanying 
documentation provided by individuals or groups within 
state/territory TAPE agencies to study initiatives in developing 
performance indicators. Findings indicated that Victoria , Tasmania, 
and the Northern Territory had undertaken relatively little 
developments although the issue was receiving significant attention 
in Victoria. Western Australia^ New South Wales# Queensland^ and 
South Australia had made significant progress in the development of 
performance indicators and systems. The Australian Capital Territory 
Institute of TAPE had a considerable history in the development and 
use of performance indicators. A review of the literature focuses on 
the following key words: planning, management information i 
accountability, and quality and quality improvement- In a final 
chapter, this publication uses an extended medical analogy to look at 
the strengths and weaknesses of performance indicators and considers 
a range of other issues, such as data quality, social justice and the 
use of performance indicators, and the development and implementation 
process. (A 78-item bibliography is included, followed by 4 
appendixes: a project network list; a list of indicators and 
collection processes; comments on specific indicators; and a list of 
performance indicators.) (YLB) 
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Chapter one: Introduction 



1988 saw a fluny of activity rating to tlw devdopmoit ami vss of perfonnance indicators, 
particulariy at the natimal levd. This work was stimulated activities oocunii^ in a number of 
Stales, but prindpally Queenslaiui, New Soutft Wales and Western Australia. Both of the latter 
States were required to report perfonnance under relevant acts. National initiatives^ coo rdin ated 
by WA TAPE produced two reports for the Australian Om^ence ctf TAFE Directors (ACTD) 
<Gokisworthy 19^a, 1988b) and a series of ^^eed and naticmally-based key performance 
indicators. 

In addition, both tl« sdv>ols and higher education sectors have shown an increasing concern with 
the use of indicators of their perfonnance. In the former case this has been reflected in the 
initiation of a project <d signikcance funded through the Department oi En^c^ment, Education 
and Training (DEED and tiie Qmference of Directors-Geteral of Educalton. It has produced, 
amongst otto* things, a series of bulletins on performance indicators (the Riporting on eduaxtional 
progress monograph series - see bibliogra];^y for a craiplete listing) as well as coordinating 
national conferences, Ae first of M^Uch was held in Sydney in 1988 (see Wyatt and Ruby 1988). 
More recently a conferaice on pofsmnanoe indicators was hsAd in Carbena. 

Recent literature firom the higher educatton sector both in Australia and overseas (particularly 
ccmttnenlal Europe and the UK) has desoibed significant devetopments in this area, particularly 
thc^ related to p^formai^ of indivklual institiitioi^ as well as a a>roideration of tiie range of 
sodo-politiral iss«ies thrown up the development and use of perfoimaiwe indicators. 

While 1988 represoited a high point in the TAFE systms' activity at a national level in Australia, 
prepress has bea> made towards the devdopment and use of indicators as planning, management, 
dedsion-making, quality improvemmt and accountability tools since tiiat period. Most of this 
activity has occurred at a State/Territoiy level however, and relatively little literature describing 
what has been going on is readily availaUe. 

Interest at the national levd was reawakened in tiie le^ up to, and following the ^)ecial meeting 
of ministers responsible for trainhig held on 2 November, 199C. At that meeting the ministers . . . 

. . . endorsed the development of itew managerial information ^rstems to meet the 
nccus of system managers at the institutio Jl, State and Commonwealth levels. 
Substantial pn^;ress towards specifying should be made by 1 July 1991. Ministers 
also endorsed ttie preparatitm of a unifcmn system of key pcitonnancc Indicators 
witti annual pttbUcaticm of eomp&risons based on these key peifoxmance 
indicators. (My emphasis) 

(Resolutioi^ of the special meeting of 
minivers respon^le for training, 
November 1990, p.4) 

VEETAC noted the !wed for urgent action to establish key performance indicators which were 
valid across Australia's TAFE systems. 

The VEETAC Conunittee on TAFE and Training Statistics is playing a leading rde in Uie 
development of these indicators, using tite 1988 work (Goldsworthy 1988a, 19S8b) as a basis. 

The present project forms part of the TAFE National Centre's response to these national arel State- 
based initiatives. The project was proposed in 1988 and approved for funding by fhe Centre's 
Board in 1989. 

Bn»dly, its aims are to examine and report on the range of indicators used by various agencies 
and describe ai^ compare the mechanisms used to select, define, measure ami report the 
indicators used. It has achieved this by: 
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■ establishing a national networic of those interested in the development and use of 
pcrfonnanoe indicators (see Appendix 1 for t}« names asid contact address^ for tlw 
network); 

■ hdping to devdop a paper relating to performance indicators for ihe VEETAC ConuniHee 
on TAPE and Training Statistics for submis^on to VEFTAC (COTIS 1991). This paper 
gatfwied information from individual State/Tenitoiy TAPE agendes about their current 
devdopmaU and use of perfomiance indicators and has formed the basis of the first three 
chapters of tiite report; 

• ^thningr reading and critically reviewing reports and a variety of other documents 
produced by State/Territoiy TAPE agendes; 

■ gatherii^ emd reviewing other relevant literature both from Australia and over^as. (Much 
of the literature gatiiened from TAPE and other sources is contained in bibliography at 
the end of this report); 

■ visits to and discussions with key ^aff conconed with the developn^t and 
imftonoitation of pefformance indi»tors in a number of TAPE fluencies; and 

■ attendbig or organising appropriate seminars, workshops or other forums relevant to this 
project. 

Nevertheless much of this report focuses on th(»e indicators which have had a measure of national 
agreemoiL 

This report integrates these various activities and considers: 

■ the current interest and cUntate in which the developn^t of po'formance indicators is 
occurring (Chapter 2); 

■ lu>w the various State and Territory TAI^ agexKies are develo|^g and using perfonnance 
indicators (Chapter 3); and 

■ what the literature both in Australia and overseas has to say about perfonnance indicators 
(Chapter 4). 

This will be drawn together in a final dtapter which will highlight and discuss the issues raised in 
the earlier chapters of the report, draw some ooiKlu»ons and suggest son^ future directions. 
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Chapter two: Performance indicators - context 
and nationai initiatives 

Introduction 

As part of the process of preparii^ a ^^per on performance indicators for VEETAC tite Centre 
undertook an examination of tte devdopment and use of a range of liuli^tors in all 
States/Tenitoites. While the study focused in those which had been ^ireed to nationally in 1988, 
other indicators in use and aate processes w ae also explwed. The present diapter therefore 
presents the information gathered for VBETAC, togettier with other information whidt has been 
ccdlected mbsequ^tly. Much of ti« fcmner infomtatkm was gatimed with tfie hdp of menibei^ 
of ^ VEETAC Committee on TAPE and Training Statistics. This as^stance and the help of thdr 
colleagues in a number of States and Territories have been invaluable in preparii^ the contents of 
this dtapter, and Chapter 3. Indeed, mudi of its content represents tiie edited content of letters 
and o^Ka documents made availal^ tten. 

The context tor the development of perlomianco Indicators 

By any measure TAFE is diverse. While the provision of vocational education programs or courses 
at a variety of levels (ranging from short courses through trades to paraprofessional and 
piofessioiial courses), areas sudi as furtter eduaition, pre-vocalitHial pn^g^aira^ labour market 
programs and trairoeships have to be taken into account. TAFE agencies are divase too, varying 
in their organisational size, scope, governaiuie and af^roacl^ to manf^ement Thus the 
boundaries of TAFE are broad find hard to establish widi any gr^t prodsion. Despite this> the 
various TAFE agencies have much in common. Much of tiUs commonality is expressed in their 
misdon ^t»nen^ for example TAFE in Westm Australia sees its mission as: 

To meet client needs for skills formation with qxmlity services which contribute to 
economic and social developn^t 

(1989-90 Annual Report, Department of TAFE, 

Western Australia, p.3) 

Typical goals of such mission statements iiKlude: 

■ To provide a broad range of relevant education and traiiung programs of quality to the 
clients of the system; 

■ To achieve the greatest possible use of tl^ community's investment in people and fadlities 
in TAFE colleges; and 

(Goals for the State Training System - 
Information Statement Number 5, 
March 1990, State Training Board, Victoria) 

■ To develop an organisational culture and practices, including accountability nteasures, to 
maximise educational output and th« achievnnent of managment priorities. 

(Corporate Review and Annual Report, 
1989, Department of En^loyment and 
TAFE, Soufh Australia, p.8) 

In NSW under the new Tedmical and Further Education Commission legislation, TAFE is required 
to present to the minister an annual corporate plan. This plan must specify obfectives, strate^es, 
budgets and criteria for assessing the performance of the commercial ajnd non-commercial activities 
to be tmdertaken by TAFE in the following financial year. 

Within the context of the three year corporate plan, the maiwgcment plan focuses on key activities 
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for fhe next year. In its draft management plan for 1991-92, the NSW TAPE Commission has 
developed seven ol^ectives, all of which have measurable outcomes and imjriications for 
perfioraiance indicators. The TAPE management plan objectives are; 

■ Provide education and training |»vgram$ to efficiently and effectively meet the State's 
soda] and industry raeds. 

■ Implen«it marketii^ strategies to adiieve an overall self-fuiklii^ tai^ for TAPE of 12% 
and returns on funds oi^c^^ accej^Ue to govemn^t 

■ In^jrove the quality of service to intemal and extmud custonvrs. 

■ Advise ihe minister on issues relating to post-oompulsoiy education and trdning, and 
policies and structures for the effectiw and efficient operation of TAPE. 

■ Inclement new n;anaganent information systems for TAFE's devolved organisational 
structure. 

■ Manage human resources e^<^vdly. 

• Implement a more dfective and ^kient framework for the managentoit of TAFE's 
financial and physical resources. 

The outcomes of dtese objectives are primarily concerned with: 

■ improved outcomes for midents; 

■ improved customer satisfaction; 

■ increased productivity; 

■ inoeased iiKome geroration; and 

■ improved management informatics systems. 

In the ACT the 1989 annual report of the ACT Institute of TAPE (p^) has> as one of its 4 key goals, 
the management of 

... the human, physical and financial resources of the Institute to entire effidoKy, quality 
and effectiveness of services provided by tl« Institute. 

Similar ^»1 staten^ts exist for the odier TAPE £^eKies. 

The following extracts indicate how several of the States aivl Territories are addressix^ the 
implications of their mission statements and corporate $pa^, particulaily in relation to the 
develo]nnent and use of manager:ient information systems and their associated peiformanoe 
indicators. 

The State IVaining Board in Victoria aims to im|nt>ve accountability 1^ (anu>ngst other thin^): 

■ [the! development of an integrated management information system and ^e publication of 
system standards; aiui 

m publication of information about the performance of providei^ through the use of standard 
performance indicators. 

South Australia has two relevant strategic priorities. These are: 
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■ To improve database and related systems; and 

■ To introduce indicators of peifonnanoe which ivill assist department, colleges, (coitral 
office! divisions ami program management groups to plan, imnage and report their work. 

TAFE in Western Australia sedcs to achieve its missicm by improving coO^ efifecfiveness and 
organisational efficiency. New South Wales has identified a range of stiat^;tes to improve the 
measurement of its peifbnnance. These strategies include tite monitoring by coU^es, coU^ 
networks (now Institutes) and industry training divisions of student outcome measures for selected 
courses and also promoting common techniques for monitoring improvement and assessing 
customer satisfaction. 

Other States, including Queeisland, are actively engaged in improving and integrating Aeir 
central stati^tteal databases in order to provide betts' managemont infonnaticHi systesi&- Indeed, 
tiie Increased emphasis on in^noved public administration in Queensland has been manifested by 
tiie introduction of a system of pto^am managraient whidt uses the inocesses of strategic 
planning, management information systems, paformance indicators md regular program 
evaluation aiul review to in^nove ^e quality of goods and so-vices, ther^ meting its 
acoountid^ty requirements better. Ihe approach is s«munaiised in a series (tf publications eititled 
'Making resuHs mattef (Que^island Treasury 1990&, 1990b, 1990c). 

Key words and phrases derived from the mission statements and corporate goals, and which are 
relevant to the developn^t and use of performance indicators, thendfore iiuilude: 

plannii^' 

using and reporting; 
quality and ^.niality improvement; 
accountability; 
^iidency and dHectiveness; 
meeting needs; and 

better management and associated information sysms (which may be used to help 
measure the achievement (tf goals aivl prtorities). 

Performance indicators must be seen within the context embraced by the key words and phrases 
listed above. However, perftnmanoe indicators are not an end in ttiemsdves, but help to provide 
the information needed to adiieve endt. Moreovo-, the development of performance indicators is 
not a strai^t-forwani emdss: the definiticm and derivation of summary statistics or the ratios 
wHch make up a numbo' of the irKSicafors can pres»it difficulties. 

If the data from which indkators are derived are not accurate (or if data elranents are going to be 
compared), and are based on different definitions or intopretations (tf a sii^ definition, there 
will be a compounding rffect on the accuracy of the indteators themselves*. At ti« national level 
those responsible for ^thering and processing TAFE's statistical data are very conscious of the 
need to improve tihe accuracy, comparability am! validity of TAFE's statistical collections. 

Work at the national level on improving the accuracy and validity of the TAFE national statistical 
cidlection is occurring under the auspices of the VEETAC Committee on TAFE and Training 
Statistics (C0TT5). One of this conunittee's roles is to develop a new national management 



Thii iaue is fitktd vp again In ChaptOB 4 and 5. 
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infonnation/statistical system (NATMI^). This syst^ wQl provide for fte management 
infonnation needs at State/Territoiy and naticmal levds, dra%ving on data from adle^ 
managmtoit ^ten» aivl eiKompass student, cjorse, staff, financial and oihsr resounds data. 

In particular, the new infonnation sj^tem will: 

■ provide consistoit, timely and accurate management da^ both within and between 
systems (at national level); ami 

■ sustain a set of national key performance indicators (KPIs). 

A project office has been appointed by COTT5 to work at national levels. The role and duties 
of diis officer indude: 

■ developing the scope of management and stati^cal information appn^niate to VEETAC 
decisions on TAFE; 

■ identifying the data concepts aiKi r^tionships which NATMISS shovld encompass, on 
dther a compulsory or discretionary basis, and define these in operational terms; 

■ developing ihe specifications for nationally con»stent managem^t information data sets to 
be used at local, syston and national levels, including key perfbrmaiKe iittiicators; 

■ developing a program for the implemratation of NATMISS, encompassing tite progres»ve 
expansion of scope, information programs, computo' systems, staff training and 
pid>Ucaticm requirenwnts; and 

■ plannii^ in conjunction with staff in Cbmmonw^th, ^te/Toritory ageiK^ and the 
TAFE National Centre for Research and Development, for ihs delivery of information from 
the varieiy computer systems whidi are op^ted by tin various colleges and agencies. 

$lm has been set aside to support these initiatives^ 

N^lonal perfomianco Indicators In TAFE • a background 

In Fdmiary 1^ a group of TAFE plarmers met in Melboiniw to om^er flie work being 
undertaken in each of the States and Territories on the devdopmmt of performance indicators. 
This meeting was coordinated by staff from TAFE in Western Australia. One of the outcomes of 
this meeting was a series of suggested performance indicators which the group considered could 
be used at a national levd for gaugii^ the effectiveness aitd effiderKy with which the TAFE sector 
achieves its objectives 

The report produced by this group (Goldsworthy 198Sa) was then forwarded to the Australian 
G>nference of TAFE Directors ( ACTD) for their conunent The ACTD oidorsed in prixKiple the 
usefuliwss of a natfonally agreed set of performaiKe indicates^ and referred die docurrant to a 
special ACTD working party. Tlw report of this latter group was tabled at the August 1988 
meeting of the ACTD and the following reconunendations endorsed: 

Recommendation 1. 

ACTD to accep't the following lutional performance indicators as an agreed set of 
performance iruiicators: 

Effectiveness Indicators 

(i) Student destinations; 

(ii) Examination pass rates (changed subsequently to subject pass rate); 



ERIC 



(iii) Sat exantfnation tates (dumged to sal^ect comptetion rate); and 

(iv) Annual graduate numbers. 



Effidencv Indicators 

(i) Student contact hours per teachii^ hour; 

(ii) Average teaching hours per fuU-time teacho? and 

(iii) Recunent cost per student contact Yvoui per course. 

Recommendation 2 

ACTD to OMlorse the iwed for an ongoing woridng group to establish the actual 
negotiations and mechanisms as direct negotiation with tlw Commonwealth will requin* a 
coordinated consistent stance. 

Reconmiendation 3 

Subject to X reronvene an af^ropiiate TAPE performance indicator group to finalise 
details and methodologies. 

The woiUng group also leoomxnended that indicators of ^d^t satisfaction, industry satisfaction 
and unmet demand be devdoped. It was fdt, however, ttiat th^ should be ref^red to tfte TAPE 
National Centre for Research and Development'. 

As a result, the TAPE performance indicators planning group met again in October 1^ undo* the 
auspices of the (then) Joint Cbmmittee on TAPE ^tistics (JCTAPES). The terms of reference of the 
woridng party were to: 

(i) Establish a starelardised methodology (induding definitions) for the collection of 

data for the perfbimaiKe indicators outUrad in recommendation 1. 

(Ii) Outline the fonnat and frequency of performance indicator information that should 

be presented at a national level. 

(iii) Outline the format of otter pcribrmance indicator information that would be 
useful as a management tool within each State TAPE system but would not 
necessarily be reported as national statistics. 

(iv) Provide an audit of ihs current ^ius of poformance indicator devetopment 
%vithin each State TAPE system and their respective requirements to implennent the 
recommended set of effectiveness and efficiency indicators. 

A report of this meeting %vas prepared (Ck>Idsworthy 19S8b). Ihe meeting also considered how to 
ccdiect the performance indicators outlined in recomn^ndation 1 aiKl examii^ oUter potential 
indicators, indudii^: 

■ student satisfaction; 

■ industry satisfaction; 



* The TAPE National Centre for Research and Devdq»n«nt has devdoped procedures for both student and 
Indu^ (dlent) atbfactton for tlK South Au^raUan Department of Emj^nait and TAPE (me Htytaa et 1991). Ibis 
vriKk has beoi based in part on work oonducied In other Sute/Tenitocte. In addition, Weston AuMralia has been 
ooouniHicBied 1^ the TAPE National Centre to conduct a pto^ in gathering inforxnation on studm desOfo^. 
Queensland (see Weelv 1990) has ooUected data on student destinations. A number of States aiui Texiitarles have 
devric^ and trlaUed tiiese and a range of odier tndicatois. For eumpl^ Queensland TAPE has attempted to assess the 
tmmet demand for Us oburaes in Streams 200CM500 in 1990 (KVFET 199(h). 




■ unmet demaiul; 

■ annual student contact houi^/equivalent fuU-tinte lecturer; 

■ participation rates (of targeted grou|:»); and 

■ student contact hours/seat of capacity. 

Sevml of Hvs iiulicators listed above have been developed and trialled in a nunrto of States and 
Territories (e.g. Weeks 1990, BEVFET 1990a). 

A particular concern of tte group was tte quality and accuracy of the basic data from which any 
managsnent infonnation ^tems and tiidr associated poformame indirators iveie developed. It 
was ^t, th^ that thexe needed to be »ib^antial improvements in tiie expecting ami processing of 
data (as well as eliminating differences in definitions of key features) if indicators were to be 
comparaUe and to £McurateIy reflect r^ty. These concerns persi^ Improvements are bdng 
pursued by means of a conqn^diensive program to establish nationalty oon^stent managnnent 
information and statistical ^tems, a task previously undolaken tl« fbrmo' Joint Conunittee on 
TAFE Statistics (JCTAFES). This work continues under COVYS with national project funding in 
1991 of $lm. 

The key Indicators at the national tevel 

A sumirary of practice with respect to the key iiniicators outlined on p^es 7 and 8 is pres^ited in 
Figure 2.1. Details about how each of the indicators is collected in each State/Territory are 
appended (see Appendix 2). 

At the CXnrS meeting in Fdmiaiy 1991 the list of ACTTD agreed indicators vras examined. Since 
ihe meeting, agreed indicators have been refirad; additional indicators were also proposed and 
omsidered. 

Subsequently a paper was developed by CXyiTS and presented to the April 1991 meeting of 
VEETAC This chapter and the next are derived very largely from that VEETAC paper (COTTS 
1991) and its associated appendices. 

The indicators proposed and agreed to are in four main areas: 

■ Client assessment (student destinations; student and industry satisfaction); 

■ educational achievement (includii^ sat ocamination rates, subject pass rate, 
course/program completion numbers, student contact Iwurs/direct teaching hours, average 
teaching hours/full-time teacher and cost/student contact hour); 

■ access (TAFE participation rates and urunet demand); aiKl 

■ commercial activity (industry funded training/total recurrent funding and a staff 
devebpment/industry experi«u?e indicator to be developed). 

The COTTS paper (COTTS 1991) makes a nurhber of comn^ts about each of these specific 
indicators. These notes are appended (Appendix 3). 

The COTTS responses to VEETAC suggested that the move towards the collection and publication 
of a set of national key performance indicators (KPIs) has highlighted the current problems which 
all Slates and Territories have in consistently defining the scope of vocational and further 
education, as well as the training sector (including TAFE), in a rapidly changing environn^t. The 
emergence of the need to encompass nteasuremenl of (if not the coordination oO training in 
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private piovidersy Uuiustiy and a range ol ottor training initiatives has magniAed ihe problems of 
perfonnance specification and measureinent using perfoimance indicators. 

Figaxe U Key pezfbxmance indicators and their use 
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Notes relating to Figure 2.1 



I. CflUected by 6 k^ia^ Training Divistons in NSW (out erf 10) 
Z. Sane data available through Examinations Btai^ 

3. lUgsnled as usdeoa atatislfc and dangemis infonnition without substantial qualificatims and notes. 

4. See infdnnatian in Appendix 2. 

5. See infonnation in Appendix 2. 

& b file moat wUkiyusadiidkatcr of ooUege perfonnance, 
Z Not fonnaOy antstttuted as a perfonnaMe indkatcr. 

8. Audita of ttie 1969 lesouree agreonents cut doubt cm ihs value of these indicators because of the manner in 
which tfiey were set a»l repcvted/aggrepted* 

9. Planned as key perfonnance imUcita^. 

10. Cunwt adlecticn and cooiputation iMthods not seen as suffictenlly reliable or accurate for use as a peffrannance 
indicator. 

II. While tte indicator marked as In ise' can be calcubted, oonsideraUe manual effort is invcdved to assmble and 
ptooeaa fee na rri ii r y data, 

12. Hoping to have feia indicator availaMe in 1991. 

13, In many oaca these data onnot be ooUected at tl^ooursetevd. They are ccdiected at a higher order of 
aggregation CDepaiiznent, School etc.) 

Moreover, ihae are significant problems within eadi Slate and Territory which are ultiniately 
accumulated at national level. Tl^ problems present a challenge for both goal definition 
and performance measurement. There arc hmdam^tal differences in government and other 
structures related to the proviaon of vocational and further education between States and 
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Territories. In addition, worit already done for the JCTAFB and by a 00TT5 Working Party has 
shown tte variation in interpretation of well defined statistical collections between collecting 
dualities and oiganisations. 

The report suggested that in order for the set of national key performance indicators to have 
maxifnuin usefulness and to allow publication witiv>ut voltin^ of exjManatoiy text to a^t 
interpretation, work needed to be undertaken to define the core populations and programs to be 
induded in fi« initial collection. This woik will also assist identification of i^es to be ladded in 
the ongoing work of the cons Woridng Party. Induded in its responsibilities wiU be the 
preparation of a schedule to expand the scope of data collection and ddinition as furth^ adjects 
of ^t>cationaI educatton ami traifung are nK>re d^ly ddineated. 

As a result, the chair of COTTS noted in his report to VEETAC that, while the proposed set of 
initial key peifbnnance indicators will be us^, C0TT5 saw the need for nna|or extension and 
enhancement However, the task of spedfyii^ a xnore complete set of key perfom»am« imlicators 
is severely hindoed by: 

■ differences ov^ the pocdved objectives of TAPE which underpin pertormante indicators/ 
or at least uncertainty as to who stould specify ihsse objectives; and 

• di^renora rdating to ihs boundaries of TAPE; for example should it indude the 

operitioiK of subsidiary companies, or oy&es^ activities, or non-TAFE mganisations 
providii^ TAPE courses? 

There is also a need to develop indicators for TAPE tiiat pennit international oompaiisons of 
efHdemy and effectiveness with comparable sectors in other countries. These difficulties will be 
increased as the data collected under Hhe auspices of COTTS becomes a better reflection of 
Australia's total training effort. 
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Chapter three: State and Territory Initiatives In 
developing performance indicators 

This diapter contains infonnation reganiing the Vey perfonnanoe indicators agreed by ACTD in 
1988. In odditioiv other indicators which have been devdoped and used by individual 
States/Teritories axe discussedr together with infonnati(m about the processes involved in their 
development, collection and use. Infonnation on the medianlsms used to collect and report the 
key peifonnance indicators is inesoited in Appaidix 2. However die discus^ relates 
piindpally to those indicators and processes used at the State/Tenitoiy level rather than the 
ctdlegelev^ It has not been easy to establish what is happening at tite college levd across 
An^ialia, alttira^ this is of critical importance to improving the management of TAFPs resources 
and the managemait infbmiation available at the college and hi^ier levels of aggregation. 

The present chapter draws heavily on letteis and/or aoomipanying documentation provided to the 
Centre by tndhriduals or groups within State/Territory TAPE agencies. These, in fliemsdves, 
provided some useful insists into Oie concepts behind the devdopment and use of perfbnnance 
indicators. In addition, visits wrane made both to Queensland and New South Wales, two States at 
the forefront of indicator deveh^ment and use. 

Victoria, Tasmania and the Northon Territory* have undertaken relatively little development to 
date, althou^ the issue is receiving significant attrition in Victoria at present Those 
Stales/Territories who have made significant progress in the developnient of performance 
indicators and systems include Western Australia, New South Wales, Queensland and South 
Australia. Because of its size the ACT Institute of TAPE is a uscftil modd of a lar^ multi<ampus 
but sin^ institution. Nevertheless it has a considerable history in the development and use of 
performance indicators. 

The developing States/Terrttorl^ 

rtoithem Territory 

The Northern Territory noted that tiie Itet of performance indicates endorsed by ACTD in 
1988 has had litfle usage there. TWs was said to be due to unreliability of data provided 
by the souiw; however it was fat that improved training in the gathering and use of suA 

data in 1990 and 1991 ^uld see the data provision and entry procedures improved to the 
stage where some faidicators can be used reliabty. 

It was suggested that the Northern Territory is in the situation where approximately 50% 
of its TAFE enrolments are in the disadvantaged category, are in small rural communities, 
or are of Aboriginal or Tones Strait Islander descent Many of the standard perfonnanoe 
indicators are therefore seen as difficult to use, or meanin^ess. Fdr example, most of 
TAFE 'courses' availaMe to Aborigines are short (4-16 hours) and reflect community-based 
educational/Utoacy requirements ratiier tiian befa^ longer, accredited and induslry-r«laled 
trainir^ courses. Thaefore die use of some indicators is fraught with problems. SmilUe 
(1990) has produced a short discussion paper on performance indicators and their 
relevance to ttedeUvery of Aboriginal programs in TAFE. It stressed the need for the 
development of agreed indicators to be used as a fbnnative test to guide TAFE's programs 
towards the achievement of desired outcomes. They therefore need to be culturaUy 
sensitive and relevant and may be different from mainstream indicators. 

In the last 18 months those responsible for gathering statistical information in the Northern 
Territory have been concentrating on persuading the cofleges, and espedaUy the adult 



» The TAFE Natkmd Centre menUy m workshops on perfonnanoe Indialore for staff In both the Northern 
Tenltary and TMmanla. 
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educators in remote communities, to sajpfAy (via enrolment, course and staff ftmns) an the 
information which will enable some of the key indicators to be extracted, and even used. 
Consequently, olives have this year been able to extis^ details on Annual Student 
Contact Hours (ASCH) per staff member, ASCH per leirote community etc PerformaiKe 
indicators are also proposed in the fundii^ model suggested for Hhe Northern Territory 
(see Nagd 1991 and Cliapter 4). 

At {ves^t much of the data i^eded to obtain performance indicators is a>Ilected in the 
form of various (student lists, lecturer contact hours etc.) and is form-based. The 
information is collected by the regional or college office for computer proce^ng ami is 
used at the regional or oc^ege le^. Mudi of tl^ information ^thered requires lecturer 
input Information is also collected about room occupancy. 

Tasmania 

Tasmania, too, reports timt it has made relatively little use of performaroe indicators up 
until now. It has, howev»', undergone major changes, being transfmed from the 
Education Departatott to beonne a section (Ute Division of Training) within the 
Department of Employn^t, Industrial Rdations and Training. 

The above chaises have initiated a major reassessment of information needs within the 
Training Divi^n and t)« wido- departn^t CMficers inwlved in plannii^ ik»v 
management information systems suggest that computoised systrans to as^t the 
personnel and foiandal functions of the new body MriU be devkoped. Hie introduction of 
these changes n^ns that it will be pos^Ie to interlink student, Bnandal and staff 
databass. These changes stwuld aUow a greater use of performance indicators 
(particulariy efficiency indicators) which in the past have not been used to fiiU effect. 

Another factor which will result in a greater use of efficiency indicators is the introduction 
of: 

■ a system of program budgeting within (he Training Division (still in the {banning 
phase); ami 

■ a fee system based on the nimiber of contact hours attached to subjects, and 
usually calculated u^g the course's position %vithin the stream structure 
(implemented in 1991). 

Tasmania's main performance indicators are: 

■ examination pass rates; 

■ sat examination rates; and 

■ aimual graduate numbers. 

Tasmanian TAFE officers suggest that, to date, their use of these indicators has been 
limited to creating specialised reports for imiividual officers within ti^ divinon or 
department Tl^re is iv> widesprrad system of formal pubUshii^ of effectiveness or 
efficiency indicators. 

Given the acceptaiKe by the ACTD of an agreed set of performance indicators, the 
respondent from Tasmanian TAFE felt that any new computerised systems of reporting 
will have to incorporate ^rstems for keeping and reporting on the full range of accepted 
indicators. Financial planning is now carried out at the sub-program level; it was tewever 
reported that a timeframe for development and implementation of (be new personnel 
system has not been developed. 
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It was su^ested titat moves to a fee*based system this and tiie indication of a 
program budgeting schema should force a greater use of efficiency irklicatore. 

Tasmania has collected information on unmet demand undertaken by colleges responding to 
a survey from cenh-al olficc. Within the college, departments fill inihe questionnaire, Qie 
information is collated and returned io <^tnl office for processing by tilie statistics officer. 
The information is tabulated at course level and a report is prepared by tiie statistics 
officer for the G>mmonwealth. Unmet demand is grouped by stream and areas of priority. 

bi addition Tasmania offered comments on a range of other indicator categories. This 
informatfon is reprodui^ umier appro{»iate brings below. 

Financial indicators 

While individual managers make use of finaiwial infomnation, it caniwt be said that there 
is any true iivlicator involved at ihe present time. It should be noted that with the 
introductton of a full system of program budgeting, financial indicators such as actual vs 
budgeted expenditure and ofst per student hour will become major indicators. 

Student Uidtcators 

While graduation rate for ooui^ is currently u^ as an indicator, as yet no data are 
collected on student satisfaction or ^udent destination. While the roles for the various 
sections within the new department have yet to be finalised, it would seem the Research 
Branch may use student de^inations in some of its future studies- It should be noted that 
such studies are likely to be limited to a rdativdy smaU scale with small ffoups of courses 
being targeted. Given the current state of flux within the department the question who 
will cany out sudi studies stiU exists^ and to what extent, if any, destination and 
satisfaction emerge as worthwhile indicators. 

Access and equity indiccrtors 

Currently enrolment data and graduation rates are used to immitor female participation in 
courses. The sanw is true for Aborigine groups From 1991 onwards participation by 
ethnic groups will be monitored. 

Business activity indicators 

Given the introductiim of fees for courses for the 1991 academic year, it is envisaged that 
income generated will become a major indicator for the Training Division. 

Staff Indtoatois 

WMle the staff ctevefopment section keeps radimentary data on costs of programs ar\d 
money ^>ent aooss colleges, nothing exists virhich could be said to be a true indicator. As 
noted above, new systems are being planned which will enable the interlinking of financial 
and staff databases. When this occurs it will also be possible to incorporate the staff 
devdopment database wiA these, or at least, given the ema^;ing capacity to download 
information, to link databases. With th^ development, the possibilities for creating a 
whole range of indicators will dearly emerge. 

Client indicators 

The Quality AssuraiKe Branch undertakes, on a very limited scale, surveys of sorrre client 
groups. These attempt to ascCTtain client satisfactfon. However at this stage of the 
reor^nisation process there is nothing being used by that branch which could be said to 
be a true indicator. It does seem likely that in the future, surv^ work will be expanded; 
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however it is not yet known what indicators^ if any, will be used. The Cuniculum Brandt 
also undolakes informal assessment of dimt needs on an oi^ii^ basis but do not use 
anytMng whidi could be described as an indicator. 

Facilities' indic^ors 

While room utilisation rates are used within colleges for planning purposes tiiere is no use 
of sudt fodlities' indicators on any wide scale. The Phy^cal Re^rces Section mates use 
of such rates on a limited basis for individual building projects, but again tte usa^ could 
K>t be defined in any way as using indicator. 

Victoria 

The Victorian State Training Board (STB) reports that one of its major focuses has been to 
improve ^dency and effectiveness throu^ut the State's training system. In pursuit of 
this aim, the STB has developed and implemented structures, processes and procedures to 
manage tite improvement of performance across Uie State Training System {STSl These 
indude: 

■ a plannii^udget process which on an annual basis relates resource allocation to 
tl« achievemoit of specific goals and objectives; 

• pcrfdmanoe agreements between the STB and individual colleges, developed as a 
result of die pknnii^/budget proems. These agreono^ detail ^Kdfic targets to 
be achieved against idoiti^ funding allooitions. I^ormanoe nwasurement ami 
monitfflrii^ are ooiKentrated on direct measumnoit of adiieveirant of taig!^. 
Each college is therefore expected to establish a review and evaluation program. 
Performance agreem»its have bea>me a legislative requiremoit with the 
implCTttntation on 1 January 1991 of the Vocsffona/ Educatwn and Tnimng Act 1990; 

■ management plans have developed by radi college on an annual basis. These 
plans encowpass all activities undertaken by tte ooll^ for the year of operatioi. 
anu idoitify objectives, imfdesnentation ^t^es and outcome Management 
plans are also a legislatiw requirement from 1 January 1991; and 

■ performance measurement processes (induding perforanatKC indicators) which 
directly m^isure adiievement of targets and monitt)r tl« implementation of 
performance agreements ami managranent plans. 

The introduction and use of performaru?e indicates in Victoria is dependent on the 
development and implementation of an int^rated set of databases which can be linked to 
provide comprehensive information to serve monitoring ami planning purposes. 
Devdopment of sud\ a system comn^nced in 1990. Continued devefopment and 
progressive impien^tation will continue during 1991. 

The State Training Board will pursue further development of performance indicators 
by undertaking a major redevelopment of corporate infomtation systems ami by 
participating in the national review of TAPE stati^cs being undoiaken by CX!)TTS. 
Already a nuinber of dte indicators are in place (see Figure 2.1). 

A project is umlerway to develop a corporate data modd. (Such |»x>jects have been 
undertaken elsewhere, notably in Western Australia, South Australia and Queenslami.) 
The project will result in the redevelopment of all infonnation s^^tems aiui the adoption of 
a set of standard performaiKe indicators. 

However, other iiKiicators not currently in use (e.g. student destination and client 
satisfaction) will be developed for use in 1992, drawing on the work ami experience of 
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other States/Territories snd the TAPE National C^tre. CoUege directors in Victoria had 
identified such indicators as being very in^xmant. 



Data <xn subject completion and pass rates are not presently gathered centrally, although 
colleges have been notified that recording those students who an? still active in their course 
at the end of their enrolled pn^iam would be required. The college directors have noted 
dte need fo^ some form of completion imtec (e.g. couise/fHii^ram oon«pleti(m rate) 
altlwtigh work will be required both at State and national lev^ to define this iiulicator 
properly. Holmes^en College of TAPE is presently conducting a study on bduOf of the 
State Training Board into the measur»nent of con^letion and enrolment. The pro^ will 
devdop a suitaUe ddiniticm of con^^on lor TAI^ It also includes case dudies 
comiucted in a number of colleges to review cunent procedures for recording completion, 
limitations of database as iivdl as a trial of amended systems. This study will report at 
the end of 1991. 

Without the measures moitioned above, tte sy^em has few, if any, meat ures of education 
uutput and quality. Victoria has concentrated so far on the development and use of 
financially-based efficiency indicators. 

At present the ^Rden^ of course ddivoy is examined by tte analysis of financial 
statistics. Victoria has been particularly active in the de^opment ^ such indicators. 
Con^»rison of unit costs between institutions of ^milar characteristics provides indications 
of possible areas of difficulties, rationalisations, savings and/or the need for alternative 
strategies. Indi^tors of efficiency re p orted as useful, include cc»t/student contact hour, 
student hours/teadio' hours (indicator of average dass sizes) and cost/teacher contact 
hour. Neverthdess it was recognised that variation between providers can be due to a 
number of factors, including staffing profile, college location, staffing work loads and 
efficiency. 

At present a range of otho* perfc»manoe indicators is used. These imdude: 

■ staff indicators (effective full-time teaching/non-teaching and permanent teaching 
staff/contract, sessional sta^; 

■ financial (e.g. actual vs budgeted expendittuie, and a range of others generated 
from financial ^ti^cal returns); 

■ acc^ and equity (negotiated targets aduevement); and 

■ business operations. 

The staff-based indicators are asscKiated witii financial expemiiture - salaries and oncosts. 
They use a time series to give an ^fecAvexiess. measure, and may be ^jedfied as a 
performance taiget for use as an efficiency measure. Information on the teacher EFT /non- 
teaching is gathered 1^ tiie College Budgets and Performance Monitoring Section (CBPMS) 
of the State Trainii^ Board's (STB) Resources Nianagement Division. It uses the annual 
statistics and finance collections. As yet unpuldished, it is used in budget discussions with 
colleges. The latter indicator (permanent vs (^ntract/sessional stafO is collected and 
reported the CBPMS to lYeasury monthly. The infonnation is used internally in budget 
discussions, but it may also be published in reports to die Victorian Parliament 

Financial Indicators are collected by CBPMS and by the Department of Budget and 
Review. They are used internally and for budget discussions. They may be reported to 
the Minister. 

The access and equity indicators examine the extent to which targets negotiated with 
colleges have been achieved. The information is collected by the Social Justice Branch of 
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the STB and the information is reported in the form of tabulations to the STB's Sodal 
Justice Onninittee, to tite STB and the Minister. It is an Intemal indicator. Other 
pcrfbrmanoe agreement targets are similarly monitored when set (e.g. for Aboriginal 
pFogruns and s^iecific migrant fHograms). 



The Gommeidal activity indicsaton are based on a series (rf n^tiated targets in the 
annual perfbimance agreements of colleges. The indicators are for internal use and 
involve both coU^^ and the Business Development Branch the State Tk-ainii^ Board. 
The indicators include: 

■ industiy-fimded training as a percentage of recurrent income; 

■ level of activity fnovided to industry in tiie {viority econonUc sectors reiened to in 
a publication which outlines the economic i^iMities of training for Victorian 
Industries. These are divided into low, medium and hig^ priority areas. Those 
with a hi^ psioiity indude aeto^aee, heivy engineerii^ oil and gast, packaging, 
tdeoommunicalions equipment and text&s. Those with a medium priority indude 
clothii^ and footwear and fainting. Those with low |»iority indude bu^n»s, 
finance and pmpaty services, construction, community services, retail and 
wholesale and puUic mlministr&tion; 

■ number of permanrait staff wlu> (1) have recdved staff devdopn^t specific to 
industiy-funded leamii\g and (2) haw partkipated in industry-funded learning 
services; and 

• the mimber and increase in co-operative projects undertaken with other colleges in 
industiy-huKled fee-for-service activities. 

The mora devolopod States/Tsrrftori^ 

Several States have been prominent in tte deveh^mient and use of performance indicators. 
These States are Queensland, New South Wales, Western Australia and South Australia. 
In particular We^em Australia and South AustraUa provided us^ back^ound 
information for this r^rt about thdr use of performance indicators. 

Western Australia 

The development and use of indicators within the Western Australian Department of TAPE 
has had a relatively long lUstoiy. As Goldsworthy (l%8c) described it 

The developn^t of p er fo rmance indicators in TAPE Western Australia nas been 
driven by a subde blend of extemal govOTun»it pressures and intemal 
oiganisational imperatives. PuUic sector managers now wax doquent on the 
merits of strategy plans, human resource nran^emoit sd^mes, managonent 
information systems and corporate plans. We all have had to team to work wi& 
an interventionist government intent on managii^ the chuiges it wants introduced. 

One of the key cudgels in this process is a revamped Hnandal Administration and 
Audit Act (FAAA 1986), Through this act it has become mandatory for all public 
sector bodies to devdop corporate plans and to report on jnogress in achievii^ 
plans u^g previously agreed performance indicators. These indicators mu^ 
show that the resources of an organisation are beii^ used effidently and 
effectively. 

(Goldsworthy 1988c, p.2) 



Hendoscm (1987a) also one of the earltest writers in tiie aiea of TAFE pofomv^ 
tfuiicatois. This eariy paper also needs to be seen wittiin a Inoa^ context of Gotlege- 
based sdf-evaluation and edumtkmal audite (KendeRon l^Tb). In addition. Hasting? 
(1988) has noted a series of concerns and issues relating to the performance indicators 
proposed or in use in Western Australia. 

At im^sent, the devdopment, nmiitorii^ as^ maintenance of perfoimaiwe indicatc»s is tiie 
responsibility of officers within the Corporate Planning Unit with the Depastmenrs central 
office. Staff in this unit have been conimis^ned by the TAPE National Centre to i»oduce 
and pilot a survey of student destination. The project will be connpleted late* this yew. 

Western Australia noted that 

■ the devdopment and use frf perframanoe indicators has be^ mandated in the WA 
Public Sector by the Hnaiurial Administration and Audit Act (FAAA) (Treasure's 
Instruction 9M). The act disting;uishes between indicators of woridoad, efficiency 
and efliectiveness; 

■ the framework for the use of performance indicators is also stipulated ftrou^itiie 
implementation of program budgetii^ ani management in tite state puUic sector. 
This follows similar lines to uSe program mana^ment structure employed by the 
Commonwealth PubUc Service. Government poUcy olives are used to structure 
TAPE'S activities intn discrete programs, ie sUDs fonnation or equity and access. 
The structure provides a hierarchy of levds fKHn a tnoadly defined fnogram at die 
top, tea veiy specific component of activity at ttie bottom. The result, as ead» 
level branches out fiom the one above, is a pyramid ^ructure; and 

■ the program management structure for TAPE is currently under revision. The 
revised structure will allow effecthre vertical traddi^ of resources and ouicomes 
from the macro policy level to the lowest operational levd. For example a traddng 
sequence in the 9dlls Formatton Program Oevd 1) could be: para-professional sub- 
program flevel 2) - engtoeeiii^ fidd of study (level 3) - oolificate level Oevd 4) - 
Certificate course in Advanced Welding (level 5). As eadi progran and sub- 
program has defined ol^ectives and annual outcomes specified, Aere is a firm 
struchire on which to anchor the perfbmunce indicators mandated by the FAAA. 

To date, the perfonnance indicators used in Western Austr^ have been limited to a small 
number of efficiency indicators (the sat & pass exam indicators are ctmsidered to be 
effictency rather than effectiveness measures) specified in college resource agreements. 
Based soldy on workload and efficiency indicators, tfwse partkular indirators gavea vciy 
narrow picture of coU^ activities. In additton, secttons of the We^em Australian 
Department of TAPE dealing with specific student groups siKh as Aborigines, migrants 
and die disabled, maintain separate databases and construct and report their own 
performance indicators and are usually rdated to the requirements of die (Commonwealth) 
funding source (cf. the Naticmal Aborfenal Employment Strategy or the Adult Migrant 
Education Service). 

However TAPE has developed a series of draft key measures which will be regularly 
monitored. Work-based teams will be e^blished to improve p^ormance. 

In many of these areas the first task will be to collect good base line data.' Having 
determined this base line, realistic taigets can then be set. In the interim, indicative targets 
based on a percenta^ improvement on current levels of performance have been proposed, 
induding: 

■ course and subject completion rates; 
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number of people from disadvantaged groups accessing TAFE courses; 



■ moven^t towards mof^ iniK>vative ddivery approadies for cunicula; 

■ movetront towards more courses developed to reflect competenq'-based training 
and modular approadira; 

■ increases in the number of staff undergoing ^f developn^nt; and 

■ increases in the numbor of staff involved in sudt services as labour market 
programs ami indii^try-fumled training. 

Finally, the Western Australian Department of TAFE is in the process of developing a new 
and integrated TAFE Learning Managranent System (LMS) (see Lloyd 1990). As a result of 
this devdopment a new range of effectiveness indicators based on student outcon^ and 
analysis of student academic behaviour have been proposed. 

One of the components of the TAFE LMS is tl^ Student-Based Evaluation System (5BES) 
which will pro\4de for the u% of performaiKe indicators in college sdf-evaluation and 
accountability. Under the provisions of the SBES course coordinators at college level tvill 
prepare annual c biannual course status reports (CSRs) which will form the basis of 
internal managen ^nnt and external reportii^. ll^ introduction of the CSRs is linked with 
the devdopmott o tte new TAFE managen^t information system whidi is currently at 
the data-model stage. 

The course status reports will provide direct feedb£K:k from student surveys and also 
indirect indicators of student academic behaviour. In ibe CSRs, course coordinators will be 
required to address t)« indicators reported and indicate appropriate recomn^ndations for 
refinement of their course structiue^ content and delivery. 

Under the proposed SBES direct student feedback would be sought at four points and 
incorporated into CSRs: 

m on application to enter course (application form); 

■ on entry to the course (sampling surveys); 

■ after completion of specific stages or core sul^ects (sampling surveys); and 

■ after %vithdrawal or graduation (all individuals »irveyed). 

The number of tlw indicators proposed to be included in tlw CSRs are outlined on page 19. 

Western Australia r^rted tiiat it uses a furtl^ indicator to those outlii^ Figure 2.1 and 
discussed in Appendix 2. Information on female^nrolled hours is collected using the 
student data s^tem and constituted as a formal performance indicator. It is reported in 
ihe department's annual report in compliance with the requirenwnts ^veming 
performance indicators undor the Hnandal Administration and Audit Act (FAAA). 
Target for tlwse imlicators have piously been set in college resource agreen^nts, 
althou^ the audit of the 1989 agreements cast doubt over the value of such iikUcators, 
giwn the maniwr in which they were set ami reported. Indeed the acting Auditor-General 
of Western Australia pointed out that, although submitted, the performance indicators for 
the Department of TAFE had not been audited for the 1989/90 financial year. 

As a result of experience using performance indicators in the 1989 and 1990 resource 
agreements, the following directions have been set by TAFE in W^em Australia for 
further refinements. Tl^se refinements include; 



■ the further dartfication of the relation^ between ^edfic peifofinaiiee indicators 
and coiporate goals; 

■ improving the construction and reporting of pexfbnnam% indicators in onier to 
maximise their value in dedsion-maldng at the college levd; and 

■ devdoping a Oiaskef of indicators, including effectiveness measures based on 
student outcomes, to gjive a more com{riete view of ccdiege operations. 

Those prc^x^ed include: 

■ application ratio (IsL preference/places offered) - all full-time and part-time 
students. It is (to some extent) a measure of vmwet demand, and looks at the 
apfdications received against a^^ilable {^ces; 

■ good standing ratio, that is, the number of students who trigger default conditions 
in the L.eaining Management System/Total No. of Students; 

■ graduation ratio (rolling 3 year average); 

■ graduate/student satisfaction index; 

■ cost per graduate; and 

■ cost per student (all full-time and part-time). 

All these indicators would be calculated to show a current value and the percentage 
variation on the averaf:? fm the pmrious three years. 

South Australia 

In 1989 the Department of Employment and TAPE released a discussion paper on 
performance indicator (DETAFE 1989). IWs paper drew, in part, on earUer work by the 
TAPE National Cfentre (Guthrie fil al. 1986, im Guthrie 1988). The paper was prepared 
under the au^rfces of the Planning and Program Management Conunitlee (PPMO. The 

introduction of performance indicators is seen as a mechanism for asses^ and improving 
'quaUty* in the South Australian TAPE system. As the (then) Chief Executive Officer' s 
letter to all TAPE staff pointed out: 

This is your opportunity to help shape the ways in which TAPE will account for its 
own pofbrmance in ei^cmdCTing quaUty learning. This is your chance to 
contribute to an important new management process within the TAPE system. 
This management process will focus on the performance of schools, colleges and 
divisions - not on the performance of individuals, as performance appraisal will 
remain tlw responsibiUty of college and divi^onal staff and be a local management 
function. 

A wide range of meetings with staff and special interest groups was sponsored to provide 
an opportunity for all those in tl» TAPE system in South Australia to understand and 
participate in the development of new approaches to college and system mana^ment and 
accountability, aiwi &eir associated performaiKe indicators. In 1990 a performance 
indicators training kit (DETAFE 1990), incorporating videos and text, was developed and 
released to assist this process. 

A recent Centre project (Guthrie and Loveder 1990) highligjited problems in accessing and 
processing student-based data, particularly in relation to obtaining reasonable measures of 
student progress, attrition and completion. DETAFE however was already weU aware of 
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these problems and has been working to create a series of new and better integrated 
databases as well as improving tlw quality of data captured at the coBege level. 

The South Australian Department of Employment and TAPE is implementing a program of 
performance indicatois to meei several iweds. These iiurlude: 

■ provision of better management information to college-based managers; 

■ provision of appropriate toob for internal acoountaMIity purposes; 

■ provision of a 'managonent overview' to the Qiief Executive Officer; and 

• provi^n of improved pulidic information for external accountability purposes. 

The devdopment of performance indicators is seen as a key step in the introduction of 
performance-based management practices throughout the departn^. The prime 
consideration is to give college-based managers information which will: 

■ assist Aem to plan and manage better; 

■ provide motivation ami enoouragen«nt in the use of poiormance managona^t 
tods and concepts; 

■ provide tools whidi will as^st management units &aros& tie department to 
compete for lesoun^ on an equal footii^ ami to reference resource bids directiy 
to dient need; ami 

■ provide tools which wiO as»st nujiagers to account for thek- use of resources. 

The method proposed for devefofrfng faidicators in South Au^ralia will be discussed 
shortly. The Department of Emfdoyment and TAPE in South Australia has noted that the 
process of deliverii^ coitrally mllated data to management unite within college in a form 
useful to fidd managen ar^ planners has provided incentive for college-based personnel 
to identify errors in data and weaknesses in date collection processes, and to introduce 
new processes to improve the quali^ of the data. As the quality of data improves, so does 
the sensitivity of the purposes to which it can be put So far, known %veaknesses in 
existing data have precluded the inclusion of 'ratfo' type date in the distributed primary 
managem^it date. However, it is planned to introduce tl«sc types of indicators in the 
near future. 

Thus no information is presented on Ae availability of indicators in Figure 2.1 (see p.9 of 
the lepcHt) for South Australia. (Neither is any information presented in Appendix 2.) 
However, date itrans and tarns from which each of tl^se indicators can be constructed are 
nmtindy collected within the department and %videly distributed for use in internal 
management processes - particularly the planning of educational delivoy, resource 
allocation and accountability. Detailed work is in progress to improve tl^ quality of these 
data ilans and to encourage tteir use. However the indicators in Figure 2.1 are not 
formally (instituted as d^iartm^tal performance indicators. 

It was report'xi that the devolution of management responsibilities from central office to 
the fieW in South Australia will lead to the devetopment of a series of performance 
indicatois which have arisen from the field. Agreed performance indicators wiB be 
legotiated and used as a basis for resource allocattons to colleges and performance 
agreements between coUege directors and the Chief Executive (Officer. The development of 
'performaiKe indicators' is seen by South Australian TAPE as a key step in the 
introduction of performance-based management practices through the department. 
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Gmsistent with tiUs policy of devohatkm of mans^gement vespottsSiMty, aad to gain 
maximunt commitment to the use of performance indicators, the responsibility for selection 
of per for mance indicators for each management unit rests widi the management of that 
uniL Guidance in the prindples of pertonnance indicatcns is provided by a unit in ^ 
Flannii^ and Systems CXvi^n in ooitral office. These principles are grounded in a 
planning and management model whidt is based on identification and satlsfection of 
client needs. 

Howem', different units within DETAFE in Soutti Australia have different dienf bases. It 
follows that tiie performance indicators in iise across the department vary from one 
management unit to anotto*. Omleed a ^udy is ja^senffy underway to investigate tt« 
performance Indicators which mi^t be used most appropriately in the student services 
area.) Nevotitdessi, especially wi^ col^e-faased units, there is ccmsideiable 
commonality. Those itons of managonent data which are seen as most useful, and which 
can be provided from central data collections have been identified, and are jnesented 
annualty to managers the dqjartm^ as an integrated set of departmental 'fnimaiy 
managonont data'. Individual managers are free to dioose which itata (if any) from this 
set they will use, and are free to use their own indicators where they judge this to be 
appropriate. 

The pimary managemrait data are collated tiie Planning and Systms Divi^on in 
DETAFE's cental office from data drawn from a number of resources: 

■ demograi^c data are taken from the Au^ralian Bursiu of Statistics (ABS) and 
frcmi ihs SA Dq»rtn^t of Envfavnmoit and Planning; 

■ industry data are ctdlated by ^h program managemeiU group (PMG) for their 
own area of responsibility. Criteria have been developed with the assistance of the 
Labcmr Market Analysis Brandt of the Department of En^loymoU and TAPE to 
guide PMGs in this task; 

■ fodlities data are drawn from the Departmoit of Hou^i^ and Con^ruction who 
have airveyed eadi departm^tal site. Some of these data are out-of-date, and 
methods Cjt keej^ng the data timdy and accurate are cuiroitly under 
investigation; 

■ staffing data are drawn from the payroll system; 

■ expeikUture data, summarising ail financial transactions made by the department 
are drawn from the financial accounting system. These are ger^rated from tl« 
records of financial transactions made by cost centres across tite department; 

■ student enrolment, student outcomes, teadiing and class data are drawn from an 
integrated student and staff activities system. The data in this system are collected 
annually from college con^led records such as class regteters and roll books; and 

■ teduiiques for gatlwring client sati^ction data have been developed by the TAPE 
National Centre for the SA Department of Employment and TAPE (Hayton £t al. 
1991). It is antidpated tiuit colleges will routtndy survey cun«tt studoits in the 
latter period of the course they are currently undertaking, past students (graduates 
and mn-completers) PMGs will gather information on the degree to which 
indusblal dients perceive thdr needs to be met. Fee-fbr-service activities may also 
be surveyed using this process. 

Managers who wish to use performance indicators which fall outside tl^ primary 
management data are respcmsible for collecting or collating the necessary information. 
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As indicated above, peffonnanoe indicators in South Anstialian TAPE are regarded as tools 
primarily for use in managonent of units to which ttK»e indicators apply. Aocordin^y, 
Planning and Systems Division in central office annually analyses the primary 
management data: 

■ by college; 

■ by program within a college; 

■ by teaching area within a college; 
and 

■ by program; 

■ by college within a program; 

• by teachii^ area within a program; and 

■ by teaching ar^ within a college within a program. 

These analyses are dien directed to the relevant colleges and FMGs for distribution to the 
managers of dte units concerned. 

The interpretation and subsequent reporting and use of this information is largely at the 
discretion of the managem^t of each unit It is anticipated that this information (available 
for the first time in 1990), will be used as a basis for educational planning, budget 
negotiaticms, negotiation of performaiKC agreemaits and subsequent accounting for 
resources and performance. 

It is anticipated that from 1991, the analyses to 'program by college' and 'college by 
program' level will be made widdy available throughout the department in a form which 
gives a management overview of educational and fiscal performance. 

Queensland 

Queensland has shown considerable interest in the development and use of performance 
indicators for some time now. The publication produced by the Program and Strategic 
Review Branch of the Queensland Treasury, MtOdng results matter gtdddines for the 
development of performance indicators (Queensland Treasury 1990c) stresses the value of 
indicators for a range d individuals and groups, including: 

• staff; 

• individual managers; 

• senior management; 

■ central ageruries; 

■ ministers; 

• clients; and 

■ Parliament and the public. 
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Other piiUieatifBis in die series (Queensland Treasmy I99fk, 1990b) advocate a systsn of 
program management (The Bureau oi Vocational and Fortiier Eduottion and Tiaining 
(BEVFETT* has responsibility for 6 inogranis.) based on stiategte plannii^ perfomiance 
indicators and periodic program evaluatimts. These processes are reflected in the 
structural fHganisation of the bureau. Moreover BEVFET is undeigtdng a process of 
devohilion and regionaUsation. The promotion of quality improvement processes is also 
an bnpoitant feature of the Inireau's curmit thrust 

Queensland TAFE has a number of groups spread across 3 of BEVFETs divisions which 
have an interest in the development and use of performance indicators. These are the: 

■ Branches of Institutional Review and Development Program Evaluation and 
Standards Review and Operational Audit Services within the Division of 
Operational Performance; 

■ Plannii^ and Analysis Branch within the Division of Planning, Technology and 
Innovation; and 

■ Edurational Researdi Branch within tte Division of Researdi and Leamkig 
Strategies. 

Their rotes are described b^w. m addition, a series of issues was identified as a result of 
my visit to Queensland and discussions with officers from these groups. 

The Division of Operational Perfoimanoe is re^JonsiUe for as5ist^g all sections of the 
department to achieve sound managerial control over departmentzd activities, to ensure 
that programs are appropriate and that they are delivered efBdently and cost effectively. 

This role is carried out by: 

■ ntonitoring the effectiveness and efficiency of departmental services and operations 
inctepend0\tly of unit mana^rs; 

■ maintainbig a qaeA for ecodloioe in progmm managemoit and administration 
throughout the departn^t; 

■ le^nding to and advisii^ senior officers on needed changes to services, 
structures and administrative procedures; 

■ evahiating cc^ege arei selected jnivate provider courses and programs; 

■ reviewing standards achieved in departn^tal and privatdy provided programs; 
and 

■ assessing and accrediting courses. 

Within the diviaon, the Imtitutunud Review and Depdopment Branch is responsible for. 

• jriannii^ and conductis^ external institutional evaluations; 

■ developing performance indicators; 

• developii^ strategies for institutional self-evaluation; 
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■ reviewing processes and outcomes of self-evaluation; and 

■ investigating ^jedfic organisational problems. 

The institutional evaluations are used to review institutional goals and assess the 
institution's ability to meet its agreed goals. In addition, the review process, whether 
conducted internally or externally, is designed to encourage institution self-development 
and improvement. It also provides a mechanism for demonstrating the effldeitty and 
effectiveness of the institution and tiw service it provides to appreciate audiences. 
Guidelines and policies have been developed for conducting reviews (Institutional Review 
and Devdopment Branch 1989a). In addition publications has been produced which are 
concerned with: 

■ external reviews Gnstitutional Review and Developmoit Branch 1988a, 1989b); and 

■ institutional sdf-evaluation (Institutional Review and Devdopment Branch 1988b, 
1990). 

Their processes draw, to some extent, on the work on institutional evaluation by Byrne et 
M- a984a, 1984b). 

The role of the Program Evaluation and Standards Review Branch includes the following: 

■ evaluation of the overall provision of vocational educational and training programs 
in the context of current and future needs and social, structural and technological 
change; 

• evaluation of the ^fectiveness of the relationship between the expectations of 
industry and the wider community and the provisions of educational and training 
opportunities; 

■ review of student assessnr^t policies, processes and practice^; and 

■ review of tiie development of vocational skills and the extent to which graduates 
acquire the competencies intended by the program designers and which are 
needed in their employment. 

In particular the branch is concerned with evaluating the effectiveness and quality of 
programs offered both by the department and private providers. However, its primary 
concern is to evaluate the expectations of industry and the wider community in relation to 
the educational and learning opportunities being provided. The aim of tiiis evaluation 
process is the improvenYcnt in the standaids and p er fo rmance of the vocational education 
and training systent. 

Finally, the Opemtional Audit Services Branch has as its major tasks the: 

■ internal audit of departmental and institutional financial systems; 

■ assessment of the effectiveness of departmental programs against specified criteria; 
and 

• 

• provision of a consultamry service to management with regard to systems 
operation, organisational structure, efficiency reviews and effectiveness audits. 

It is tterefore concerned with conducting financial and compliance audits, together with 
audits of efficiency and program effectiveness* 
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The Planning and Pcrfoimance AnaiytU BraMh is invcdved in ooUectfitg a ra^ge of 
pnfonnanoe information and data and oonductii^ a first pass analysis of these data and 
informatioa Others (for example brandxes within the C^peiational Pefformance Division) 
conduct more detailed analyses of the data and infonnation. The branch co-ordinates the 
production and analysis of individual coll^ and divisional development plans. These 
plans indude ^tegic targets and otho* indicators. Examines of i^ peifonnance target 
areas include: 

■ utilisation of physical capacity; 

■ full-time teacher utilisation (teacher contact hours/fuU-time teacher); 

■ proportion of recurrent expenditure raised from external sources by college and 

■ cost per student contact hour- 
Other reports relatii^ to operational efficiency are published (BEVFET 1990). They relate 
inputs to outputs and are genially quantitative, comparative measures of service ddivery 
and unit costs. These permit a rang? of pofcnmiance indicates to be devdoped ai^ 
consolidated at the State and college levd& Efftdenty indicators used indude: 

■ net and gross recurrent cost/student contact hours for all breams; 

■ average da^ ^ze for all streams (defir^ as the total ^(tent ccmtad Iwurs/ total 
teadier contact tours); 

■ average annual contact hours/full-time teacher (comparative data are also 
provided between years 1988-89); and 

■ student contad hours/seat for all streams (facilities use). 

Other indicators indude enrolment growth rates in both full- and part-time modes, and 
shident contad hours, the perooitage change in teacher contad hours aiul tire ratio of full- 
tisne to part-time teadiii^ hour& All of these data are expressed at the cdlege levd and 
can be used for comparative purposes on an institutional or yearly basis (Le. comparing 
1988 with 1989 data). However it is acknowledged that tite data gathered concentrate 
heavily on effidency nvasures. Moreover, it has been notei (see BEVFET 1990b, p.23) 
that 

■ valid comparisons can only be made between colleges operating at sintilar levels of 
output; 

■ new colleges, smaller colleges and specialist colleges will generally have higher 
operating costs; 

■ iwn-practical classes (e.g. management) generally have lower operating costs; and 

■ effidency perfonnanoe measures only indicate activity levels. Further analyses of 
exceptions are necessary before any qualitative judgements and decisions can be 
made. 

This indicates the potential difficulties which might be caused by the uninformed use of 
performance indicators. 
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The biandi is also conoemed with gathering dita and other infonnadon which reflect 
elements of the quality of outcome s and service, induding graduate destination surveys 
(Weeks 1990), unmet demand (BEVFET 1990^) and stuctent satisfaction (using a 
questionnaire to the previous year's graduates which is distributed, processed and reported 
by State Ottke). Sudt studtes are used to to ludge the ^fectiveness and quality of tte 
oiganisation. Indeed there is a s^ni^icant intoest in is»ies of quality aivi its measursnnent 
for impTOvonoit purposes in Queoisland 

In ^ditton, infonnation on staff salaries and training is beii^ compiled usii^ AES training 
survey fonns. Hie^ data are being collated and will be reported by State Office. 

The Education Rraeaich BianA has had a relativdy limited role in the dev^pn^t of 
perfomunce indicators. It jnovides a oonsidtancy service to the branches and bodies; it 
monitors work being conducted elsewhere and acts as a cl^ringhouse for this information. 
Recendy it has been studying the use of pafonnanoe indiators relevant to distance or off- 
campus provision. 

ftew South Wales 

New South Wales, like many other TAFE agencies, lacks systems whiCh provide useable, 
timely and accurate managoi^t data and other inlramation fm a rai^ of levels within 
its organisation. However tiie responsibility for the collection, processing and reporting of 
perfonnance indicators feUs to the Corporate Planning Division. Information jm>vided by 
the Division of 

Information Systems and other bodies within the corporate structure provide much of the 
data necessary for calculating a range of performance indicators. 

Uke Western Australia, TAFE in New South Walef is required by an act of Parliament to 
report on its performance. The current processes and systems bdi^ developed and 
prc^ressively implemented will ensure tiliat performance can be measured against 
determined objectives and outcomes. 

In tl» NSW public sector generally, management information systems, performance 
mea^ren^t and customer consultative processes are still in a devebpmoital stage. For 
such a large State as New South Wales, the challenge for TAFE is to create an integrated 
planning, p^ormaiKe monitoring and reporting system to meet a range of internal and 
external needsw 

At present considerable effort is bdi^ put into the devdopn^t of management 
information systons; financial, human and admini^tive, to ^ppoit the collection, 
processing and evahiatfon of perfbrmanoe data. This integrated ^stem will assist 
manageis at tte local level to use iikUcators to plan, monitor and evaluate dteir 
performance. Tte aggregation of local data will meet corporate planning, perfomumce and 
reporting needs. 

The Corporate Planning Division has a co-ordinating role in collecting, processing and 
reporting corporate perfonnance indicators and actively supports colleges, networks and 
training divisions with the developnwnt and collection of lo^ imiicatcnrs. 

In 1990, NSW TAFE collected and reported 13 corpcnate perfonnance indicatbrs. The 
indicators were collected primarily to meet annual reporting requironents aiui to provide 
TAPE'S executive staff with an overview of fhs organisation's perfonnanoe at a global 
level. Listed below are the performance indicators reported in tte 1990 NSW TAFE 
Annual Report: 
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Gndoate ntunbos - nttodxr of students in major award ^mrses who were eligible 
for an award. Rqwrted 1^ faculty and sdiool and for srJected onurses in 6 
Industry Training Divisions (TTDs); 

Couise completion nites - nun^ier of students airolled in major award courses 
over number who completed over 4 year period. Reported for sheeted courses in 
6 ITDs; 

Aven^ subject ounplctlon rates fax couraes - number of students who enrolled 
in su^ects in major award courses over the number who completed. Reported for 
selected courses in 6 ITDs; 

Cffinmunity participation rates - percentage of selected populations 15 years and 
over who participate in TAPE. Repotted by State, metropolitan/country, and by 
age and u^ng data extracted from cen^ data; 

Enrolment rates for targeted groups - percentage of enroUees from taigeted 
groups such a$ women, noneng^ speaking bacl^iround and unemployed youth. 
Reported by /acuity and educational program; 

Cost per student contact hour - total salaries arul consumables e)q)enditure over 
number of student contact hours. Reported by school- 
Student satisfoction wittt course content and delivery - surv^ of current studotts 
enrolled in selected courses in 6 ITDs; 

Graduate satisfaction with oouise omtent arul delivery - wrvey of recent 
graduates from selected courses in 6 ITDs; 

Graduate employment arul further trairdng outcomes - survey of recent graduates 
from selected courses in 6 ITDs; 

Industry satisfaction with trairdng - survey of selected industry representatives in 
6 industry groupings; 

Staff attrition rates - for permanent employees only. Extracted from personnel 
records; 

Days lost per 100 staff doe to irulustzial disputes - for teaching staff only. 
Extracted from persormel records; and 

Percentage of salaries and wages spent on training - extracted from general 
records. 

To continue the development of Ae system in 1991, all colleges, networks and training 
divisions will be required to monitor and/or collect a standard set of local performance 
indicators. These indicators are con^tent witfi the organisation's ov«aIl objectives and 
are directly linked to performmce objectives, outcomes, and targets specified in individual 
coUege, network and training division managemoU plans. By integratii^ the monitoring 
and reporting of indicators witfi the managenont planning process, indicators wiU assist 
managers at all levels to plan, monitor, evaluate ai^ improve their performance and 
services to students. 

Listed bdow is the set of local indicators to be nwnitored by all colleges, networks and 
training divisions in 1991: 

■ Student satisfaction - aU colleges administer survey to a sample of students; 
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■ Siib}ect completion n»u - reported by sid^ect for each odlege, network and 
training divlsion; 

■ Subject retcntloi.1 rates - reported by subject for each college, network and training 
division; 

■ Avtfage subject complctlan r^es for courses - reported by course for each 
college, network and trainii^ division; 

■ Enrolment rates for targeted groups - reported by targeted group for each college, 
ratwori( and training divi^on; and 

■ EGO d^ - report fomtat yet to be determined. 

In 1992, the set of local indicators is likely to be expanded to iiKlude: 

■ actual versus budgeted ccpendlture; 

■ graduate enjoyment and further training outcomes; 

■ industry satisfaction with training; 

■ non-con^leting studotts' perceptions of TAPE; and 

■ cost per student contact hour. 

While the development and use of indicators may lead to a better corrdation between 
plans and outcomes it was acknowledgied that it is difficult, initially, to set reasonable and 
meaningful targets. Moreover, those providii^ data to the syst^ needed to be persuaded 
of ^e benefits to be gained by in^[»oving Qie quality of data and otiner information 
gathered. 

The ACT 

In a brief paper, Reporting Jtey performncf indicatm tc gooenmmt the ACT reported that it 
is develo]ring a range of key peiformaiKe indicates as an integral part of Ute ACT bud^t 
process. This process requires all ACT govemn^it agencies to report on perforrance 
against nominated indicators for each program. These are published each year in the ACT 
gov^nment budget papas. Performance indiiators also form an important part of the 
ACT Institute of TAPE'S busine^ fdanning process. 

The indicators include: 

■ access • knowledge of courses, accessibility to facilities and entry to courses; 

■ participation - numbers and mix of students, including those presently 
disadvantaged; 

■ successful educational outcomes for students; 

■ quality and diversity of courses; 

■ responsiveness to the economic and soda) needs of the ACT community; 

■ collaboration %vith other ACT education, training and industiy agencies in pursuit 
of common goals, and in relevant national endeavours; and 
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■ achievement of 6b)eclives within budget. 



Key performance indicators are specified to government within three sub-programs: 

■ education delivery; 

■ education services; and 

■ corporate services. 

Tliese ixuiicatofs are essential to the Institute's mission to 'devdop a skilled and flexible 
workfoite'. (Outlined in Af^iendix 4.) 

As indicated in Figuie 2.1 (see p.9) ttie ACT also has sy^ems in {dace A^hich make use of 
all the ^ectiveness and effidency indicators listed for eompariscm, at the national Icv^. 
These indicators;, and other variations used by Commonw^th go ve r n m en t agencies (such 
as the Department of Rnance ami tte Grants Gommi^on) rdy totally on the valkiity and 
accun. of flte nationally collected 'sdected TAF£ statistics'. 

As pai r '}f its ongoing analy^ of sdected TAPE ^tistics over some five to six yearsy and 
in its suomissions to d«e Gmunonw^th Grants Commisaon Inquiries^, ACT TAPE has 
attempted to compare and omtrast Ite opoations wittt otho- Stat»/Tenritoiies. Such 
analysis has, however, been frustrated by tiw incon^^rat coUecticm methoddt^es and by 
consequoitial unrdial^ data for some States in iStte national collection. Other States have 
akx> been attempting sudt analyses. 

EvCT wititout oonsid^ing important questi<ms rating to the objectives of TAPE and its as 
yet umlefliwd boumiaries, the above comment serve to illu^te the degree of caution to 
be obserrad in the devdopn^nt and use of performance measures in both intrastate and 
interstate analysis. All of these issues are to be «ldressed in the VEETAC ^nsored 
national managonent information and stati^cs ^tem project described in Chapter 2. 

Notwithstanding the difficulties, ACT TAPE is attempting to estaldi^ international boich- 
marks, or performance comporisons> with compaFable tertiary in^tutions aiKl ^ten\s in 
other countiies, e.g. New Z^Jand and tl^ USA. Tt^ irareased marketing of TAPE courses 
off-^ore - eqaedally throu^ fiill fee paying overseas students - makes such international 
comparisons increasingly important. 

Perfonnance indicatois in use 

While there has been coredderable attention gi\^ to ihe how, the why and tl% wherefore of 
performance indicators Uiere are surpri^^y few exan^?Ies which can be cited of their practical 
t^. Moreover the ma^rity of the infoimation that can be r^dily collected is at the 
State/Territory rather than tte national level. It also relates particularly to the selected national 
key performance indicators ami otl^ available information, all of which can be used by TAPE 
central offioes in their dealings with colleges ami other groups. Relatively little infonnation is 
fredy available at the college level ^nce much of these data are tied up aitm in confidential 
(dannii^ documents, resource agreen^ts or reports of institutional evaluation. 

Much work has been done both ftmnally and informally on the development and use of 
performance indicators for evaluation, planning and (ntt>re re<^tly) managen^nt purposes. Little 
of it however, is in a form where it can readily be obtaii^ since it has not been formally 
published. The development of management infom^tion systons his also been local and based at 
the college levd, although in some systems (for ratamfrfe New South Wales) certain databases are 
managed centrally. Other States (for example Queensland) have been workii^ on the 
development of a common, but decentralised martagement information system designed to take 
account of the data needs of colleges. 
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There «{^)eais to be a lade of vesdUy availaUe tofimnatkm about the devdopment and use of 
colIege4as«i management infonnation systems and performance indicators. Therefore a 
particulariy fruitful area of future work ^vould be: 

■ a siin{^ manual on how to develop and use p erfor mance indicates; and 

■ a project to devdop a relatively sin^e computer4>ased mam^oan^t information ^r^em 
which met the needs of colleges (and which could be readily adapted to address further 
data needs as identified) but which incorporated within them the information needed for 
both State/Tenitcny and national data reporting. This development process would need to 
tate account of those performance Indicators mo^ likely to be u^iil at the college level 
since it is inevitaMy at this levd or below Otat da^ ooUection begins. 

Perhaps the most notaUe exan^le <rf the public use of poformance indicators is their inclusion in 
the annual reports of the Department of TAPE in New South Wales (more recoitly the NSW TAFH 
Commission}. While a range of geiwral ^tii^cs and imiicators are r^rted (e.g. numbCTS of total 
enrolments over a time soies) other statistics are reported as well, including: 

■ completion rates for a ninge of courses and subjects; 

a employnwnt statistics and satisfaction rates in major subject areas (e.g. hospitality and 
tourism and electronics, computing and telecommunications courses); 

■ participation rates of particular groups; and 

■ utilisation of space and costs of consumables. 

This information is particulariy informative for those within and external to tl% TAPE ^tem in 
New South Wales. In addition it has been reported (Henderson pers. comm.) that having 
perfitmnance indicator data available has helped to win a greater proportion of State resources in 
times of general cut backs ^ince need can be dentonstrated. 

Performance indicators will also be reported for major areas of activity in BEVFETs annual report 
for 1989-90. Similar informati<m is available in the SA DFTAFE annual report Simple 
perfcmnanoe indicator data appears in the anmial reports of the ACT Institute of TAPE The CHfice 
of Tertiary Education in the Northern Territory publishes selected TAPE statistics. 

There is a need for more attention to be paid to the development and use of indicators - 
particulaHy at the college level. However, an increasii^ amount of informatton is available about 
the development and of performance indicators at the State/Territory level also, although it is 
most often concerned with technique and process rather than use. There has been relatively little 
informatton pia>lished up until now about the performance of individual TAPE systems, whether 
in ti^ own ri^t, in comparison with other s^tems in Australia, or intonationally. However, 
ttoe may be an increasing demaivi for irWUcators of this type to be devdoped md used. 



Chapter four: Performance indicators - what 
the literature says 

In Chapter 2 some of the mission statements and corporate goals of a lai^ of TAPE agendes were 
examined. woids which en^iged from this proce^ included: 

■ [banning; 

■ managjem^t Information; 

■ accountability; and 

■ qtialiiy and quality improven^t. 

These will be used to focus the review of the literature presented in the present chapter. In 
addition I will re-examine the d^nition of pofomiance indicators and also consid^ some of the 
issues and proUems %vlth performance indicators and tiieir use as highlighted in tl^ literature. 
Some of tiiese issues nvill be con^oed in greater detail in the final chapter. 

Psfformanco Indicators redaflnsd? 

A previous Centre publicaticm (Guthrie 1988a) examined a range of definitions of the temd 
performance indicator dra%vn from the work of Henderson (1987a) and n^odossin (1987). 

Indicators were seen as; 

■ suiTO^tes or proxies; 

■ substitutes for thmgs that cannot be n»asured directly; 

■ quantitative or qualitative; aiKi 

■ mea^res whidi hdp to caphue the esseiKe of what is happening. 

PQfonnanoe irulicators may also be though of as a communication device, an analytical structure 
for conveying relevant infomnation. An essential feahire of its definition therefore will be the 
manna- of its use in Ute communication proces (Flndlay 1990). 

However Vroei)enstijn and Acherman (1990) poim out that, despite attonpts to define the meaning 
and function of performance indicators^ confusion still exists. Bamett (1988) and Hattie (1990) iwte 
the danger of performance indicators as correlational statistics because the indiators may not bear 
directly on performance (the example Hattie used was random breath testing and a particular 
blood alcohol level as a direct correlate of driving competency). Moreover Vroei)en^in and 
Acherman also suggest that: 

■ pecfple attribute different functions to perfomonoe indicators, e.g. acoountalrility versus 
improvOTient (and tiUs han^xrs tite developmmt of a common definition of tlw tenn); 

• they aie readily transferred into standards (although Hattie (1990) poinb out that most 
indicators are a statement of r^tive quality and a norm rather than a statement of some 
standard); and 

■ the users of performance indicators have different objectives or goals (this, in part, is a 
reflection of the political use made of performance indicators). 
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However tl^ repres^tt diagiwstic features or tools, which can be used initiany to help assess the 
state of orgai^tional health or 'wellness'. This analogy will be devdoped further in the 
following chapter. 

During the review of tite literature other definitions of performance indicators have emei^^, for 
Qcample: 

P^^mnai^ indicators are the means vised to cdlect, on a ^r^ematic basis, data about the 
performance of programs . . Iwhich] enable information to be generated about 
performance against goals . ■ . [and] relate U> ends not to rmsns. 

(Queensland Treasury 1990c, p2,6 and 8) 

They are 'indicative' ratl^ than 'conclusive' and may provide information about effectiveness, 
sodal justice, operational and outcome efficiency and standards of service (Queensland Treasury 
1990c). 

Hopkins and Leask offer the following definiticm: 

A perfornurae indicator is a stateimnt against whidt achievcn^t in an area or activity 
can be assessed; tl^ are also usdul for settii^ goate and clarifying ol^ectives. For some 
performance indicators, a brld statenwnt is aiffldent; for otiiers, the statement should be 
more specific and refer to supplen^taiy prooKses which would g^ve a meaaire of depth, 
quality and/or commitn^t in the particular area. For the purpose of school 
improvement, p^fomiai^ indicators should reflect a ^tlKsis of . . . aims and be 
constructed in such a way as to provide ^gnposts for development 

(Hopkins aral Leask 1989, p.6-7) 

Hattie (1990) has pooited out that, as correlates, poformance indicators are only partially related to 
outooR^ they are symptoms, vital sigi^, used in diagm^ aiKi dedsion^nakii^ but can ^sily 
become the foicus of attention rather than the outcomes thonsd^^ (that is, they become fhe tail 
that wa^ the dog). Hattie also refers to other definitions of performance imlicator^ in the 
Utonture where tiiey. 

■ are bendunarks for measuring progress or regre^on over time or diffierences across 
geographic areas or institutions at one point in tin^ (Dunnell and CyUnighlin 1^7); 

■ i-educe a complexity of fudgn^ts to a simple objective m^isure; and 

■ are used to hdp in assessing impact, discerning elective practices, infonning decision- 
makers, explaining causes, nvmitoring standards and tr^s, forecasting future changes, 
stimulating and focusing effort, defining objectives and ensuring accountability (after 
Cuttance 1989). 

Th^ are (as Findlay 1990 points out) a mesats of ^fdifying the complex information that 
underpins management's decision-making. Backmann (1%7) notes Chat performaiu£ indicators 
have two aspects - a technical proce^ (how they are constinited, validated, etc.) and a 'hearts and 
minds' exercise (which relates to the way in which they are proposed, introduced and used). This 
latter aspect has omsiderable political connotations. 

Cave £t al. (1988) have given some attention to how performance indicators can be d^cribed in 
technical terms. The description includes: 

■ type of indicator (output, efficiency, effectiveness, quality, etc.); 

■ relevance (Does it relate to organisational goals and mis»on?); 

■ ambig;uity (Is a high or low value of the indicator imambiguously favourable or 
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unfovouraUe?); 



■ dteat proofless (Can tie indicator be manipulated? - very often multiple imlicators are 
used so that 'deating' on one will show up on anoti^); 

• cf»t of collection and availability of comparative data; 

■ level of ^gre^tion; and 

• rdation to other inlicaturs (Is it a unique 'measure' or one of a number of 'measures' of 
the same attribute?). 



Hattie (1990) has suggested a range of other technical criteria, including: 

■ coherence (the balance between indicators in reflecting the essence of what is bdng 
measured); 

• dqiQviabUity (valid and reliaUe for comparative rurposes wiUtin a ^ngle time frame); 
and 



■ durability (valid ami reliable over time). 

M(»t definitions assert that performance iraiicators are goal-related (Dochy et ^. 1990) and used 
within a wido* context of planning, providing information, intended outcomes, evaluation and 
review, and improvement (anl, of course, iwoountaMlity). 



Porfoimanco liKUcatom In planning and managemont 



Their role in ptanning 

Many oorpc»ate (or strategic) plannii^ texts link the planning piocess to performance 
indicators (Cunan 1^). Beginning wiUi a charter or mission statement, an environmental 
scan is conducted wh^ examines tiie context wiUiin which tiie oiganisation operates. 
Oiganisational objectives are then set whidi need to be dear, oondse and limited in 
number. They are also rdated to key areas of activity (for example, program areas in a 
program managemott process). 

(Soals or aints are then estaMished which are measurable, followed by strategies or 
programs designed to adtieve the goala Perfonnance indicators are spedfted and are 
designed to determine the success or otherwise in meetii^ the goals (the ends) • and for 
monitoring the adequacy of the plan itsdf. Such a sequence of activities ^ts oomfortabty 
with a strategic planning appfdsh as propc^ed by Cope (1991). 

The process of program management is discussed in greater detail in the series of 
publications provided by tte Program and Strategic Review Branch of the Queensland 
Treasuiy (Queensland Treasury 1990a, 1990b, 1990c). Tl» program management system 
they propose is made up of: 

■ a strategic plan for the implen^tation of relevant aspects of government policy; 

• a program structure which is consistent with the strategic plan and which has 
goals framed in terms of desired outcomes; 

• related nnanagen^t systems, the key one bdng a system of management 
information; and 

■ a regular evaluation and review process whereby: 
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(i) performance against goals is iramitored on a regu^ basis; 

(ii) ccmsideration is g^ven to alternative iniwvative n^ans of adiievii^ 
dq>artmaital ^»Is; and 

(iii) the continued appropriateness of the departn^'s purpose, goab 
and strategies is reviewed; revisions are made and resources 
reallocated where necessary. 

(Queendand Tieasuiy 1990a, p^) 
They believe pn^^am management can assist managers by providing: 
« a comnKm franKMrorii for planning, resource management and evaluation; 

■ r^ular and systentatic assessment of perfcmnanoe of piogrdms against tiieir stated 
goals (usii^ amongst other things, peiformaiK» indicators); 

■ a basis for developing suitable supporting management information sy^ems; and 

■ o];^rtunities to engage intonal and extonal stakeholders in a dialogue about 
p eifoj miance. 

Curran identifies three differoit types of indicators: structural, process and output. Ashe 
points out 

'Structural indicators' use inputs as surrogates for outputs. lypicaUy, managenwnt 
groups will publicise structural indicators because input factors are almost entirely 
imder their control - inputs sudi as the quality of the staff they hire, or the output 
capacity of th^ installations or, perhaps, the ratio of ^dents to staff. A 
stnictaral indicator is, at best only an index of &e caq>adty ^ ttie organisation 
to perfoim woilc rather than an index of the woric ttiat is actually done. (My 
emphasis.) 

On the other hand, 'process indicators' evaluate conformity to a given standard of 
performance but not the adequacy nor the ooirectness of the stai^ards ttemsdves. 
These indicators assume that those activities which must be performed for the 
organisation to be effective are known, and can be known. Structural and process 
indicators are often used as regulatory medianisms, but r^ulatfon of the 
structures or processes within organisations can have negative effects on outcomes. 

The third type of indicators, 'output indicators', unfortunately do not necessarily 
provide a true measure of performance dther becau» tiiey rdlect not only 
organisational performance but also uncontrollable external factors. They are also 
inherently difficult to measure and perhaps nowhoe more so than in the ddivery 
of ^xxls or services like education, which has outromes vahied diff^^nfly by 
different sectors of the population. 

(Curran 1988, p57) 

The different emphasis of different groups on the cSwioe and interpretation of indicators 
and tiie problem of the correlation bet%vesi the pCTfcmnance indicator and the actual 
reasom for ttie apparott level of performance are important points. Many of our present 
indicators are struchired, but there appears to be a wish to nwve tow«^rds more output or 
outcome-oriented ii^iicators - despite their inherent problems. 

Their rote in management 

Public sector organisations such as TAPE agencies are increasingly caught between 
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properly accountii^ for tl^ use of ctmsolidated revanue fm fimdii^ their activities 
(which has tnuUtionally meant a rdatively hi^ level of cential contnd and accountability 
and thdr incr^sing maricet oriottation (givoi die deregulation of tiie trainii^ market and 
the increasing competition TAFE faces from other providers of training). This has lead to 
TAFE colics adoptii^ an increasingly sovice-related pospective and offering courses 
and other pn^rams of ^dy which have been cuslomi^ to meet Hwk various dioite' 
requirements. College directors are now less administratoFs and n«>re dearly managers. 

Incrrased autommy is necessary in order to con^>ete in this more maricet-oriented 
owironment althou^ as HOfner (1987) points out, a balams needs to be struck betweoi 
the 'state' and the 'market' forces acting on oiganisations in order to produce an 'optimal 
mix'. However inaeased oompetitivsKSS may be restricted or preve nt ed if practices or 
regulations which will enaUe greater freedom of action (e^. tiie i^t to hire and fire) are 
not allowed, or practices or regulations are kept in fdace whidi unnecessarily restrict a 
manager's range of options. 

Whiki some (e.g. Sing^ 1988) belteve that performance indicators may be used as 
mechanisms for increasii^ centralisaticm and control (as iiKieed Uwy can bel), otl^rs, (e.g. 
BcnTnanns^al. 1987) ai^e^ tiiat they offer a mechanian for providing increased freedom 
and sdf-regulation with a braad frame%voiic of acoountsd>ility. Tl^ en£d>le governments to 
issue fewer prescriptions ex ante while registali^ the results ex (Bormanns e| al. 
1987). They can also be used to develop a greater degree of institutional tran^>arency 
(Hafho- 1987). 

Neverttteless the fear diat the developn^t of a set of indicators %vill lead to ^tem 
ossifteation once introduced is a v^ i«al one. In the UK, reports midi as Obtaimng better 
vabie from furtho' ^ucatiim (HMSO 1985) dealt di!^>roportionateIy with fte issues of 
economy aiul ^Rdoicy. If adopted unquesttonably sudt indicators provide messages to 
those managing organisations (and those implementing, using and interpreting the 
indicators) about what is considered to be important. Thus it is a matter of the weighting 
accorded ^di of the indicators |»t>posed during die plaiming and management process^ 
whidi is, in turn, a reflection of, or n^ssage about, t)^ relative importance of the feature it 
is designed to monitor or measure. 

Hiifner (1987) points out that German and Austrian univosities have noted that a number 
of indicates are focur^d on odlecting information about operating irKiicators (i.e. 
compliance with standard values for ^ce, annual staff teaching hours, staff /student 
ratios, etc.) which are a reflection of parameters set 1^ ^)ecific planning and steering 
mechanisms. In Au^ralia ti«se have been enshrined in industrial awards whidi guide, 
and possibly overrestrict, die ways in whidi oi^ganisations can deploy their resources. 
While guidelines and the resultant indicators have advantages, they also enshrine input 
meamres while (possibly) igmnii^ important indicators of outcome. Thus die 
developmoit of a onnprehaisive range of iiulicators whidi r^y attempts to reflect the 
goals of an organisation, and provide a reaa>nable measure of goal or tai^ achievement, 
is difficult. Essentially it gets back to the question raised by the FEU (1989) as to whether 
indicators are a devdofmiKital tool OR a measurii^ stick. There seems to have been a 
move away (in tl» literature at least) from an initial concern with effidency, to an 
incTKisii^ concern with measuring quality and effectiveness (in terms of ttw extent to 
which desired outcomes were achieved). 

It is generally acknowled^ however that these sorts of indicators are far more difficult to 
develop. 



Their role in resource allocation and deployment 



Indicators may be used in die process of resource allocation. Industry and award 
restructuring, together with increased devolution of financial responsibility in most 
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Australian TAPE systms, are pladi^ increasing pr^sure on tiw traditional budget 
processes of incresrtottal changes to the allocation of resources. In broad terms rules 
have stayed tl« same and fundii^g decisions 'tinker at the edges^. Nagel <1991) reiers to 
this as ^historical incrementalism' iie, last year's budget plus or minus a HO. Ruttoford 
(1987) aptly describes Ais process as 'equal miseiy for all'. Many are unaware of tte 
historical precedents and processes on which funding has been based, and their current 
validity may also be qu^tionable in the light of omtextual chaises. 

The TAPE National (Nagel 1991) was commissioned to produce a furalii^ model 

for TAPE in the Northon TerriUvy. (Other States Aenitories are also addressing the issue 
of resource allocation and fundii^.) The applicatton (rf such a model would altow 
resource allocations to be influenced more by current needs than precedents (si^gesting an 
increased role for jdanning), and which owourages the pursuit of excellence (quality, 
efficiency, impoitsmoe of Hve custonwr, output/outcome rather dian activity), better 
mana^Qonent and the coltecticm of required data accuratdy aiui in a timdy manner. It was 
also seen to assist dedsion-makir^ and related resource allocation and enable tiu>se in 
power, and in tte wido* community, to percdve that TAPE repr&ents value for money, 
^afai, tfiis suggests tiie need for adequate data gatl^ing and a rai^ of perfomnance 
indicators. 

Nagel suggests the need for a simplified funding nwdd so that ti« mass of financial 
de^ is removed, but whidi allows trends and inconsistencies to show up, tl^reby 
assistii^ in dedsim^n^dng. While he brieves scmte form of student omtact htswrs 
measure represents a gra»rally accepted proxy ka output or activity,- 1« recog^tises the 
need for better, but 'more esoteric' nuasures of output He also recognises the danger 
inherent in the b^ef that more is better. Imaginative managen^t whidi irareases 
efficiency is thus discouraged. 

jappinen suggests that o(»t data are very heterogeneous, and without unambiguous 
ddiniti(ms of costing variaMes, cost data canrot be r^dily compared. This» in turn, 
suggests considerable cautim is needed in 0idr use as con^nrats of performance 
indicators. E)qE>erience in Australia (see CXTTIS 1991) confirms these concerns. HOfner 
(1987) has iu>ted that if oiganisaticms lack control ova* sudi features as finaiKe, 
admissions, staffb^ levels etc. tl^ are not encouraged to Utink strategically. 

Recent work by Lagunzad (1990) on measuring the utilisation of facilities in TAPE is also 
rdevant at both the State/Territory and institutional levd& As he points out, governments 
at both State and federal levds have been pk^ring post-seomdaiy institutions, including 
TAPE, under increased pressure to achieve higher levels of efficiency and effectiveness in 
their use of facilities. It is also useful in pUmnii^ future ^ce requirements. However 
most systems are frequency-based and ad hoc, and both the methodolc^ and resources to 
cmduct such studies have been lacking (Lagunzad 1990). 

Presently a lack of widely accepted utilisation standards exist This has inhibited the 
publication of utilisation data for fear of misuse or misint»pretation, although present 
indicators show that TAPE'S usage of its facilities is better than that of higher education 
and comparable with Hiose of Uie further education section in the UK. Guthrie and Bone 
(1989) briefly considered facilities' utilisation in their evaluation of Regency College on 
behalf of the Department of En^loym^t, Education and Training. This college uses a 
computer-based room all(xation/tinw-tabling system 

Ttieir role In evahiatlon and review 

Performance indicators can also play an important role in institutional evaluation and 
review. Queenslarai TAPE has developed self and external institutional review procedures 
(see Chapter 3). Such a process is now required in Victoria. Henderson (1987b) also 
produced a paper on ite subject. 




The F^utiter Education Unit in the UK (FEU 1^) has devdoped a coUege-hased audit 
apiRoadt materials which advanced the work of Ae Audit Commissioner (HMSO 19S5) 
and the recommendations contained in the Department of Education and Science report 
enmed, Mm^bigaSl^^fidaUfy (PES 19BSI In New Zealand, Southland Fdytedmic 
(1990) has developed a ]»t^Fam evaluation strata which indudes the pioduction of a 
im^gram fsoffle (indudii^ a iutnd»er of pg fon nanoe indlcattns), data on a»ts and a series 
of questionnaires for students, staff, graduates, employers and the program's advisory 
committee. Early work by Byrne (1984a, 19S4b) in South Australia provides much 
material which oould be used in an institutional audit and review process. 

Their introduGlton 

Kamis-Gould (1987) has noted some of the i^es sirrounding tl^ introduction and use of 
performance indicators. Three phases are suggested. These are: 

■ hostile - They can't do this to us!; 

■ asudous - Are Ibey really goii^ to do this to us?; and 

■ co-operation - Togethra* we can improve the system and assure rewards to 

deserving providosb 

Frackmann (1987) has noted 3 different stiategtes used by organisations whidi are being 
subjected to the introduction of performance indicators. These are: 

2. The strategy of d^ereiveness: The oiganisational unit under consideration tries to 

hide ii\fonnation and prevent external review and insight This aim is best served 
if no information at an r^^udii^ the unit is generated. This strategy is successful 
as long as tiie organisational unit is not punished for withholding information and 
is not suffering (eg. financially) under the prevailing conditions; 

2. The shtttegsf of incomplete aUabanOion: The (Hganisattonal unit under consideration 
partidpates voluntarily in efforts to gather iniicmmtion concernii^ the u^ The 
effort is organised under the auspices of tesf or 'j^f whidt will have 'no 
consequences', vs^tever the results of the information gathered. This results in 
maldng it unfeasible to collect information in die first place, a task of little use, 
with few consequences and results. This strategy is successful as long as the 
pressure for dtai^ is not required bodt sides, and dtar^ is not identified as 
having advantages for bodi sides; 

3. The strategy of shaping the dtange: The organisational tmit under ooi^eration 
{Rovides information that is assimvxl to be advantageous for its own muvival. It 
puts forward aiggestions for the indicators by which it wants to be judged. It 
suggests channds and utilisaticm of information, fbllowii^ more the id&H of 
autonomy and self-regulation than die model of centralised governing. This 
strategy seems to be necessary if chai^ cannot be preverUed. 

(after Frackmann 1987, p.l52) 

Often tfiese ^teg^ are a reflection of moves, usually by outside forces, to incr^se the 
accountabiUty requirements on organisations for the resources they obtain and use. A 
range of approaches is possiUe induding: 

■ fosterii^ a spirit of self-r^;ulation and accountability; arui/or 

■ using informatton externally to rratrict or enhance available resources - in other 
words to introduce tangible consequences resulting from the performance 
information gathered; and/or 
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■ using indicators as a meai^ of festering constructive dialogue betwe^ those 
within and outside the organisation. 

The intnxluction of performance indicators has caused disquiet in some TAPE agencies, 
particularly where their introduction and use has been seen as central office inspired and 
focused punitively, and nttst particulaiiy in rdation to allocation of resources. This is 
contrasted by the approach in South Aietralia (see Chaj^ 3), whidi has aimed at 
develo{^ng and fostering an informed debate wher*y the perfonnance and management 
informatioA needs of all parties can be considered and resolved. In all cases it is preferable 
if co-operation is soug^ and all parties are involved in shaping chai^ 

Man^ienMiit tnfonnailon 

Perf(Hinance indicators rely on fiie accuracy and appropriateness of the data whidi undopin them. 
The pi^sent pressure to ctevdop nationally acceptable and StatE/Territory-hased systsne of 
perfonnance indteators has led fo the desire to improve the quality of die intomation and data 
gathered widifai their le^ective systems. However colleges providing the data have had very littie 
^ake in improving data quality, seeii^ its provisimt as a tiresome obligation ratter tfian something 
of usetotivem 

The inaea^ng devolution of management responsibility means that, at Ae college level data are 
inaeasii^ly seen as something to be wed rather than reported elsewhere (see Chapter 2). This 
has led, inevitably, to an increased interest in its accuracy at this level There is aiso concern to 
develop common definittons (and consistent application of these definitions), particulariy if 
om^stfisons are to be drawn. 

In short there is an increased de^ to make the data more useful and useable. Initiatives are 
taking place at the national level and a number of States are redeveloping their date models (see 
Ouipter 3; OOTTS 1991) aflo- reoonsiderii^ their information needs. For exan^l^ Ltoyd (19W) 
shows how the Western Australian Depaitaioit of TAPE is moving to adopt a Learning 
ManagenKnt System (LtASi which puts student outcomes at fhe centre of a collegers oiganisation. 
It is proposed to phase it in over a 4-year period. One oDmponoit of the system (the student- 
based evaluation system - SBES) will use a range of indicators to provide managen^t information 
(the LMS and SBES are discussed in Chapter 3). 

Moreover, data need to be analysed rather than Just coUected. TWs suggests the need to describe 
what data are required in the 1^?^ of oiganisational goals and objectives. How these data and 
other information wiU be analy^ used and reported in devdopii^ an appropriate data model 
and management information sy^em needs to be determined. Management information, like 
performance indicator, only provide part of fte information a manager needs in order to make 
decisions. 

The advent of computer-based systenns of data storey and manipulation and retrieval at a 
reasonable cost has made the introduction of integrated management infbnnation systems and 
their related performance indicators more fea^ble. Neverthdess most Australian TAFE systems 
have been left with a rdic of unrelated databases developed for iweds in the past Now the need 
is for greater integration. 

Moreover many colleges have not had the advantage of ready access to a management infonnation 
system which assists dedsfon-making at flie college level - let atone at tl« level of schools, 
departments or individual teachers coordinating courses or subjects. Such systems are needed if 
accurate and reliable data are to be obtained for use at higgler levels. However it is difficult, tf not 
impossible, to devise a management information system which will comprehensively meet every 
information need (QueenslaiKl TVeasuiy 1991a). 

Ideally, hoivever, Quecndand Treasury suggests that such systems should: 




■ be directed toward tiie siting and monitoring of departmoital goals and ot^ectives; 

■ fadlitate ttie devolution of authority and re^ponaOrili^ 

• be focused on a rdativdy small number of key performance indicators which require 
regular reportii^- These indicators should show the department's overan performance 
against both longer-term strategic plans as wdl as short-term taigets; 

■ be able to provide information both on resources usage (injmts) as well as outputs and 
outcomes; 

■ be abte to produce timely reporte in readily oomprdoen^Ue foraiats with sufficient 
flexibility to meet diangii^ requimnents; 

■ be cost effective; and 

• draw on the same data sources used by line managers for operational decision-making. 

It is also necessary for the data to be aggregated across work units and through management 
levels. In other words, tite infonnation colle^d has to be us^ul to a variety of groups, and at a 
number of levels. 



Hopkii» aiMi h&sk (1S^9) describe the aj^noach adopted by tte Newcasde Education Group to 
the use of peifuim anoe indicators in evaluating the technical and vocatkmal edi^tion initiative 
(TVED im)gram in the UK. They view the tee of managemott information qrstems in terms of 
ongoing pnwedures of quality oontrcd. In this way vaM and meM information is fed in at all 
levds. Subsequently the variois management lev^ iMed to inter|»et and malffi use of the data in 
the U^i of their own spedalised knowledge of situaticms within tiie uniKs) which they are 
managii^. GT\is is where the qualitative Infonnation comes in!) 

Using the information system enables strengths and weaknesses in it to be revealed. Thus the 
users can assist in any revisions needed to the date collection {m>cesses or instruments. The model 
is |»eEented in diagrunmatic fonn below (after RtzGiUxm and Hazelwood 1988): 

Figure 41 A model for a quality infonnation system 



DATA BASE 



Inputs and outputs 

-4 

pe r form arce indicators 



Review variables 
-> 

hypotheses 



MANAGEMENT 
SYSTEM 



O>nsultatioiv e.g. 
quality drdes 
ledes^ of data base 



QUALITY 
INFORMATION 
SYSTEM 



Thus data, managen^ systems and practices (e.g. quality ci des) are linked to provide a quality 
formation ^rstem. The issue of quality will be addressed shortly in this chapto-. 

The final fector related to managemoit information is tfie quality of the infonnation itedf . A 
variety of authors in die higher education sector (for examfAe Vroeijensti^ and Adierman (1990)), 
and in the further education sector in the UK have expressed concern about how the academic 
performance of an individual or a department might be indicated. As Iheodossin points out: 

The British education system has not been notorious at institutional level for sustaining a 
valid database on whidi to take dedans. Traditionally fbere has been veiy little (his 
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emphasis) attempt at fdnnal (ie. organised, systematic, regular) data collection. 

*^ (Theodosdn 1987, p.19-20) 

The same would be true for mudi of education in Australia - including TAPE. In addition, while 
definitions and weightings for indicators need to be proposed and agreed to, there will also be 
considoable d^te over die technical de^ of g»th»ring, processing and wdg^ting the data if 
intrastate (let al<me national) comparaUlity is sou^t 

The VEETAC Committee on TAPE and Trainfaig Statistics (see CXJTTS 1991) has expressed its 
concern over Uie quality and comparability of the data gaftered both within and between sy^ems 
(seeC3iapter2). This problem is magnified when data are consolidated, ratio-based measures 
devdoped,andsystansGon^pared. The i8si» of the quality of data is tted up with its immediate 
usefulness to those ^Wio ooUect it In flie past little use has been made of such information for 
management purposes - althou^ the potential was always there to develop and implemoit 
management information sy8ten\s and their associated perfonnancs indicators at a local level. This 
issue will be picked up again in fbs final diapter. 

AoeoumabUity 

I have already noted the uneasy rdationship between the use of indicators for development versus 
measurement and control, and for accountability versus s^regulation, performance monltoilng 
and improvement These traisions are real, and reflect the balance between and strengA of the 
political forces acting to control or to shape what an oi^anisation does - its actions and its 
priorities. It is, in a nutshdl, the balance between external control (or a measure of it) and sdf- 
i^ulation. This issue was discussed in an earlier section of this chapter. However, if 
accountability is 'bottom up' conformity and consistency are the essential elements and imply 
commitment to con^>arison linked to external oontrol (Theodossin 1987). Top down accountability, 
on the ottter hand, is about diversity and customisation. Onnpariscm in iStmse circumstances is 
meaningless. It therefore depends on who is the focus of aocountaWUty - those 'above', or those 
'below' who are served by the oiganisatfon (Le. its custcnners or dioits). 

Performance indicators are used for acoountaWUty because they are readily understandable. They 
appear to be simple and convenient to use. However thdr apparent simplicity can mask a 
complexity which underiies thdr definition, derivation and interpretation. Fbr example, as KalecW 
(1989) points out 

The shipidest thing is not to calculate, the second most stupid thing to do is foUow blindly 
the resulte of oik's calculations! 

If not carefully conceived they may only have marginal relevance to the feature of the performance 
which they have been developed to indicate. They are also one of a number of potential 
mechani&ms of accountability and control 

The move towards die use of performance indicators may be Treasury-inspired (see Hopkins and 
Leask 1989, Queensland Treasury 1991a) and coincides %vith, or is part of, the movement towards 
program managonent and budgetii^ favoured in Amtralia recen^ at both federal and 
State/Territory levels. It relates to goal- or target-oitenled approaches to management However 
die measure of external control being exercised is determfated by die extent to which the taigpts 
are imposed ratiierdian negotiated. Altemadvdy they may be set die oiganisation being 
'me^uied' i^df and then agreed to an external r^;ulating body. 

Some cf tiw more recent Uterature seems to be heralding a swing away from the use of 
performance indicators to serve die needs of acoountabiUty, particularly diat which is 'bottom up'. 
Inacasii^y diey seem to be focusing on diem as 'tnp down', and as measures of quaUty and the 
quality of outcontes (and for quaUty improvemCTt). Perhaps it could be termed accounting for 
quality. 
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Quality 

The concept of quality in relation to a TAPE agency, an institution, a pn^ram or course of 
instructifm or training can be defined by its degree or grade of excellence. Feigenbaum (1983) 
describes quality in the following way: 

Qi^ty is a customer determination, not an engineer's detenninati(m, not a marisetiiig 
determination, or a genoal management de^mination. It is based upon the customer's 
actual experieice with the product or service, measured against his/her requirements - 
stated or unstated, conscious or merely sensed and always represents a moving target 

(Feigenbaum 1983, p.lO) 

Thus quality is a i^tive tom, dspandent on the attributes chtraoi (and excluded) by cu^n^rs 
and ^ relative weightii^ they give the various attributes. 

M<aeov& Zeleny (1991) suggests that the quality of any product or service depends on the quality 
of ttw underiying pnxsss of its design, production and c^veiy. Bemuse it is a relative texm, 
m^isures of quality are soi^t Hie process of evaluation r^&cs to the asessment (tf mait or 
worth, and so many of the techniques used to evsduate iaititiillfms (see Byrne £ia|,> 1^ and 
various publications of the Institutional Review and Development Branch witiiin Queensland 
TAPE) o: programs (see Guthrie ^gl. 1986, 1988; Queensland Treasury 1990b) can be used to 
determine the degree or grade of exodlence - their quaUly. Perfbraiams indicators (see Guthrie 
1^8a) represent another mechanian whidi is potentially helpful in measuring qualify, as long as 
ttie attributes of quality can be specified aiui a variety of means used to m^snre them. However, 
it has been sugg^ted that they are, at bes% only partial operationalisations of quality 
(Goedegebuur«^aI. 1990), because quality is a multidimensional concept (Westeriieifdin 1990). 

It is also important to have a concept of appropriate quality rather than, necessarily, 'high' quality. 
Whilst the two are not incompatible, a 'hi^' quality program, school or institution may not be 
efadent and cost-effective. It may not meet tlw needs of those it serves. In short, quality has to 
enibiace the law of diminishing returns to ensure that it is not maintained or improved by an 
uiweasonable use of the available resources. 

Needless to say the oiteria used to determine quality can compete, or the quality dimensions 
chosen may be at a variety of levels, because perspectives diffo-. In short, what constitutes quality 
in the eyes of one party may not have the same importance in those of otl^. It may not even be 
a f^evant attribute at that level. TWs gets back to a range of questions pwed by Paardekooper 
and Spee (1990) such as: 

■ What standards should be used to detect differences in quality? 

■ Who should Judge quality? When? Udng what techniques and approaches, and to what 
purpose? 

• What role is there for a proper information system in this area? 

■ To what extent should these judgements be verifiable and comparable, and what weight 
should tlKy carry? 

Quality might also be linked to achievement of goals (Westerhei^en 1990). 
Cope has defined total quality as: 

... a philosophy of continuous^ constant, impassioned improvement cveiywhere, by 

evervone in tiie oreanisation all the time. 

^ (Cope 1991, p.80) 
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Quality and its improvemont are therefore functions of the entire oi;ganisation. Hiis Bts in wdl 
with the ooncei^n of Total Quality Managen^t (TQM), that is, a managemoit philosophy whidi 
tttiltees tools and techniques fon 

■ osntrol of quality, 

■ planning of qu^ty, and 

■ continuous improvement, 

in everything which oi^ganisations cany out It is an attitude to worldly towards continual 
impnmmaA in quality and fnoductivity, at all levels of oonunocial and intustiial oiganirations, 
in order to meet customos' needs at eadi stage of the prooesa Perfoxmance indicators can play a 
vay important role in this process because measurement is central to the methods 

The concept of TQM suggests that everyone has a custoxner, or a range of customere, whose needs 
have to be served as well as pos^le. Howeva* (as Zdeny points out) quality has often been 
pushed out by a survey data saturated (informed) producer rather than pulled in by an integrated 
and empow^ed cummer who has defined quality in tarns of fSxe fttmss (rf a |m>duct or service 
for their use. In slwrt, quality can be pn)duo»-- rather than cu^mersfaiven (as it poliaps should 
be). 

Thus Zeleny believes quality management or in^rovmoit fnocesses need to integrate tl« 
customers into processes rather than separating them from them. In other words it is concerned 
with gatlwiing infbnmation about what is happenii^ rather than what custon^s, and ottera, say 
is happoiii^. To control the quality of outonras, the organisation needs to in^ement qrstems 
which control the processes involved (Hayton et al. 1989, pp. 18, 19), 

If performance indicators are to play any role in such a quality improvement system tlwy must 
free it not constrain it (It is much easier to talk about quantities in education than its <piality, see 
^^h (1988): lippinen (1987) doubts whether quality can be measured with indicators.) It also 
means ttiat re^slaiuse to chai^ witMn an organisation must be reduced to a minimun\ and any 
processes which indirectly or even directly reward and encourage resistance to change eliminated 
(for example, outdated or inappropriate performance indicators). 

In many cases the promotion of chuige will involve the renv>val of traditional ctedcs and balances 
because, in nnost traditional oi^^misations change is pushed out from the top and encounters 
restrictions furtho- down which are numovus, strong and persi^oit (Zeleny 1991). On the other 
hand, in an imfnovement-oiiented management system, for example the integrated process 
managen^t (H'M type), change, quality etc. are gei^ated at bw^ and more proper levels of 
contact pulled in and propa^ted upwards encountering (perhaps) only sin^e w^ ai^ 
tmsustainaUe resistance at tte top. 

Another principle of TQM which is relevant to in^lementation of performance indicators is: 
"people work within systems" (Hayton ej al- 1989, p-14). DenUng (1986) has stated that 85% of 
problems in an organisation can be attributed to the system and 15% to individuals. 

Hafde (1990) points out that fhz debate over performaiKe indicators often pronwtes discussion of 
deeper and more important issues (for example. What is quality and how do we go about 
improving it?). They can be (as Theodossin 1987 su^ests), tite first step in establij>;ang a viable • 
quality control or assuraitce syston. 

Dochy et al. (1990) suggest that quality assurance is a continuous process of control. Quality 
mea»irement is a snap-shot focussing on shorter-term outconies. Quality assurance is a cyclical 
process which includes three independent phases: 
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■ monitoring (servii^ as an eariy warning syston to trigg»' comprehensive a^essment of 
the causes of d^teiCTdes that are disco^^red); 

■ measurement (including a linked nun^yer of performance indicators with high validity to 
detect defldendes); and 

• im]m>vement 

Ho%vever, mo&t quality impioveiKnt systems t^ to undoemphasise tiie latter two phases 
(Dochy A. 1990). The cyde of quali^ assurance is presented in Figure A2. 



Figure 42 The cyde of quality assurance 




The process is not dissimilar to the prwesses of external and self-review considered briefly earlier 
in tills chapter. Neverthdess it touches on the notion of a culture which promotes, maintains and 
in^noves teaching quality (see Sisser 1990) and is related to regular reviews of subjects, courses, 
teaching metiwds wd resourees, quality of anvase d^veiy, stoiff devdopment ai^ apfnrdsal 
procedures and client satisfaction. 



fnm what has been said so far, it is dear that 'quality" is not easy to define or to measure. It has 
already been stressed tiiat Uie iwtton of quality is a matte" of opinton and tl^rdore its 
measurement is determined by those elemoits which individuals or groups choose to identify and 
to en^hasdse. Therefore (^^ons about quality in education and trrining may lead to ttie issue 
beccnning '^political'. Neverthetess perfonnanoe tedicatore have an important rde to play in 
enhandi^ the quality dd»te, and in focussii^ on and hdjring to measue these features whidt 
help to tdl us about tite quality of ou* processes and outcomes. 

Other IstUM hH|hllgm«l by tho HtPraturB 

Many of the i^ies arising in tlie devdopmott aiKl use of performance imlicators have already 
been discussed. A number of the nwre important of these will be considered in greater detail in 
the foltowing chaptff. Hiis section will coiKentrate on tiiose issu^ identified by tite literature 
which have so far reodved little att^ntton. These i^es are as follows: 

■ tte use of titt proce^^ of peer review in assessing quality; 

• performance appraisal at the individual level; and 
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■ a range of issues and concerns raised by Singh (1988) regarding performance indicators 
and their use. 

These issues arise not so much directly from tiie performance indicator literature itsdf; raUtov they 
are a reflection of Ute wid» context in whidn peribnnanoe indicators are devdoped and used. 

Peer review 

Peer review traditionally has been used in the education sector (and particularly so in 
higher education) in ttie referee system for academic journals. In tV i^resent context 
therefore; the term peer review is used somewhat kxi^dy (Goedegebttuie 1990). 
Indeed the whole publication edited by Coedegdniuie and his colleagues pay considerable 
attention to the issues of evaluation, peer review and its role in the development and use 
of perfonmance indicators Certainly the term ^)eer review' Involves human fudgemsit 
which may or may not be informed more or less objective databases. (Amin^ the more 
'objective' data are those derived from performance indirator&) 

Who are these peers, however, is open to considerate speculation and cM»te. In UK 
universities reviews by peers occur at the course or departmental levd. They may also 
take i^ace in a vaitety of fwms - from individual to syst^ (depavUng on who Oie peers 
are!). A range of literature is available on the subject (e.g. Roe and McDonald 1^, Moses 
and Roe 1986). Peer review may be used at the course levd to ^vide feedb^ to those 
involved in devdopii^ and in^Ionattii^ particular courses. Such a proce^ (usli^ 
Course Review Committees) has been em^^iyed by TAPE in South Australia (Guthrie and 
B(Hte 1989). Finally, peer review it may be used at the indiv^ual level in a process of 
dinical supervisicm (see Guthrie 1988b) or, at a more senior tevd for the review of 
individual senior managers (Mageean 1990). However, in general, Australian tertiary 
education lacks a culture of evaluatiim OSourice 1^), altttough Hattie (1990) examines tlw 
use of performance indiators in asses^i^ quality of depaitmoits of education in 
universities in AusMdia. 

review Is seen as having high content validity (Goedeg^uure 1990) because it 
can be used directly to assess qii^ty. Ho%vever its inherent ^ibjectivity is seen as a 
disadvantage; obviously however, sudi a process can nevs* be fulty automated 
(Paanlekooper and Spee 1990). External review also complements internal self-review 
(V:oeijenst^ and Adierman 1990) and the processes of review 1^ peers on die one hand is 
not incompatible with the use of performance indicators on the other. Indeed the 
Queensland Treasury (1990a, 1990c) proposes the use of perfomianoe indicators AND a 
process of evahiation or revtew (199(S>). The innportant issue has ceased to be Shoukl peer 
and other reviews be GCHKiuctedT' The question now relates to the technical details of how, 
when, and where titey are done. Second, the review activities and finding? need to lead to 
action. Hiiidly in Australia the answers to the questions of what information is needed in 
this review process?, when?, how?, and from whom?, are still being resolved. 

Peiformance appraisal 

For the most part the performance indicator and performance appraisal debates have 
mnaiiwd separate - often quite <j^iberately. Appraisal is seen Gohn^n 1989) as the 
identification of strei^ths and weaknesses; and the devdopment of programs of action that 
will enhance tte strengths and remedy the weaknesses. Appraisal, like review, can occur 
at a variety of levels • from an fatdividual to a system and beyond. The project conducted 
to develop itulicators of dient satisfaction for the South Australian Department of 
Employment and TAPE (Hayton gl ai. 1991) was very conscfous of ttie soisitivlty of the 
data it and otho- processes like it could generate. In courses with limited owolments and 
at small iitstitutions, it is likely that the level of performance of individual staff could be 
identified. The issue is whether this information shoukl be available to manago^ how it 
should be used and to what end. South Australia (Hayton e|^. 1991, p27) has kept the 
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issue of perfoimance appraisal at a local mahagnnent tevd. 

The recent review of TAPE teacher preparation and dev^pment (Hall £t ai. 1991) noted 
that apfnraisal was an issue in tead^ devebpntent. Some groups were ooiKen^ about 
the opportunities for its abuse, the criteria upon which apjnaisal would be based and its 
processes. Others ivrted tlw potoitial usefulness of ihe process in iniim>ving individual 
and departmental performanw, ami in inoea^ng individual professionally 

As Bouite has noted in relation to hig^ education: 

One mark of a profes^on is that is has standaids of practice and performams which aie 
maintained tiirmig^ regular scrutii^ conducted by fellow practitionos. Academic work 
has jTOgressh^ expoienced in most of its attributes in the last half-century other 
processes of professionalism ... But in some fields a pre-professional notion of 
performance has been siiiprisingly tenacious. 

(Bourke 1986, p23) 

In hig^ education in the UK the debate has linked the notion of indicators of 
perfmnanoe to those used for individual opjpr^saL Some staff (see Rutherford 1987) 
would prefer not to see the formalisation and legitimisation of cunent informal processes 
while others were concerned that such appraisal occurred behind the scenes and was based 
on criteria that were unknown and hence far from dear. In short, they bdieved processes 
need to be more open to scrutiny. It was noted tfiat while improvement mi^t be 0)e 
functim (by providing help ami retraining) ttiere was a fear that it would be med for 
judgement, and to help identify departments and individuals to be ^tionalised'. 

Appraisal processes may involve staff, peers aiui students (Johnston 1989). Moreover, 
appraisal process^ have been linked to the probation and promotions qrstems (Rudterford 
1987). However, such piocesses require that resources be allocated (or reallocated) if the 
outcomes of the appraisal process indicate that dianges are needed. How such appraisal 
systems wouU work - whether at the individual level or above - will require fuith«- 
consideration by those proposii^ such approaches as well as those who KiU ultimately be 
appraised. If nothing else, it will make currently informal systems r\7>fe formal, with 
re^nsiMlily for any developmoit or improvement resting with the individual or, more 
likely, jointly with tiie to^tutkm or oiganisation. Howev^, one-way a{^>raisal will always 
invoke strong opposition (Rutherford 1987). 

It is likely that the two issues (performance indicators ami performance appraisal) will 
become increasingly linked. Nevertheless, there is considerable emotion in both issues, but 
the latter (being more personal) exerts a more immediate impact 

Other Issues 

Sii^h (1988) has highlighted a range of issues, and the second order (and often 
unantidpated) consequences associated with titc use of perfomiance indicators in 
education. Some of titese (not already covered before) are listed below. He suggested that 

they: 

■ are a displacement ability - a self-serving activity for corporate managers, rather 
than a serious minded effort to emure accountaHlity (and) to improve education . . 

a (possibly) have adverse effects on students, teadwrs and teaching, and quality; 

■ ere the public sector equivalent to the corporate sector's profit and loss accounting; 
and 
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■ are Indicatois of actkm and of the estabUshmenfs for sJ^t or ben^ Their 
have a symbolic use r^aidless of the validity of tiie ^tistio. They may be used 
to (^Tscuie rather 6ian illuminate reality. 

These and a range of otter issues haw been raised in Chapter 3 and ihs present one. 
These trends will be drawn together and considered in the final chapter. 

Summary 

The literature and the various State /Territory TAFE agendes see the introduction and use of 
perfbrmanoe indicators as ii^talde. This not a bad thii^ they are already pranoting dd»te 
on Issues sudi as quality, acoountaUUty, lesource altotion, plaiming, management and tiie 
quality (rf data whidi underpins many of Htvs indicators already in formal or informal use 

They hai« considerable potential to be useful management tooAa at a lange of levds and to inform 
the debate about the efficiency and effectiveness of TAFE which has hitherto largely been fuelled 
by rumour, innuendo and gossip whk^ (unfortunatdy) has also oftoi been exaggerated or untrue. 

The literature genmlly supports the introduction of indicators; howevr? the cautions expressed 
are no Itmger related to whether or not ttey shoukl be introduced and xit^^ Rather they relate to 
what indicators should be introduced, for what purpose and how validly they reflect reality. The 
debate now is concerned with tiie tedmicalities and with %vinnii^ hearts and mimis; the issue of 
whether or not to have them is dead, lliey are toe now; tite die is cast 

A range of conclu^ons can be drawn from tiie review of ^ litoature. It is dear that: 

■ (here is still oonsider^^ variance in tiie literature over the d^nition and |»%dominant 
purposes for devdopb^ using and reportii^ indteators of psfoimance; 

■ they are now perceived more property as donents of a managemoit information system 
am] tiie management process rattier than isc^ted tools; 

■ they can be used to promote d^te and increa^ professionalism^ bottt in terms of 
educational and mana^mait practice. Neverttidess thoe is still a m^sure of scepticism 
about their validity and ultimate usefulness; 

« indicators are progressively bdng seen less as tools of 1x>ttom up' accountability; rather 
more OS promotii^ 'top down' accountability, with its associated ooiwem for meeting 
needs and fnomoting quality; 

■ tte move towards tiieir role as a managem»it tool has been accompanied by an irereased 
concern with their use in resource (and particularly budget) allocation. This is particularly 
so as traditional views ai^ govoitment foraiing niodels are broken down; 

■ the dtoice of iiuiicators to indude or exdude in the set of indicators chosen provides 
messages about what is considered inqxmant Ihoefore the dtoioe and use of iiuiicators 
needs to be based on their appropriateness rather than their availability and the extent to 
which they can individually or coUectivdy provide a picture of what is actually 
happoiing; 

• if indicat^^s are not dtosen aiui used wisely iS^ey will restrict rather than foster diversity 
and goou practice; 

• a range of indicators has to be used, vfhere possible, in order to obtain the most valid 
indication of performance; 
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attention to the quaUty of the data upon which indicators are based is a preying need both 
in Australia said ovoseas; and finally 

a sense of ownership by those involved in collecting and usii^ the information needs to be 
fostered if performance indicators are to be implemented and used effectively. 
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Chapter five: Issues and conclusions 



Introduction 

TMs publication has attempted to draw together infcmnation about how pofbnnanoe indicatofs are 
being imptemented and used by TAPE agotdes thiou^ut Australia (Chapter 3). This has been 
considered in the context of TAFFs peicdved ndssitm and coiporate goals (Qiapt^ 2) and in the 
wider context of government policy and the literature from both here and ovaseas (see Qiapters 1 
and 4). This final chapter attempts to identify and discuss key issues as wdl as readi some 
condu^ns about the state (tf play in tiie introduction and use of pofoimanoe indicators by TAPE. 
Specifically this %vill be undertaken by. 

■ lookii^ at seme of Hhe strengths ami %veaknesses of perfcumance indicators (and draw 
some conclusions about its directions) by using a medical analogy; and 

■ ccm^derii^ a rar^ of otiier issues (for exan^ data quality, sodal justice and the use of 
perfonnance indicators and the devdopment and implonentation process). 

Porfonnanco Indlcatofi - a iredleBi analogy 

The problon with many analogies is that if they are simple they succeed; excessive analysis 
means that they break down* This wOl be tiie case here, but in doing so a number of imipoitant 
conclusions and issues about performance indicators - and the directions in which debate and 
practice seems to be movi>v; in this fidd - will be drawn out 

When we visit our general practitioners with medical complaints, they are benefited in their 
diagnosis by having previous records about us as well as loiowledge of our personal trails (a 
qualitative background). Cotafaily they receive firom us impressions and tedings (again, 
qualitative data), but they also have a range ctf hard data avaHalte too (for oaasxfAQ, pulse rate, 
Mood presmre, tenq)aature, etc.) and a nund)er erf oU^3r readity available performance indica^vs 
to help them with their dia^to^ At a second level they have certain other tests availaUe, some 
of which can be perfbrmed at their surgery, others whkh need to be sent to a qyedalised 
laboratory (e.g. blood tests, bkxkl »igars, cholestool etc.). Keferral to specialist doctors or to 
sophisticated (and often extremely expensive) testing are other possiUe optiora. 

So it is with performance indkators. Some of them are simj^e and useful in helping with a first 
pass diagnosis; others are more sophisticated, requiring more (and Increasin^y) complex 
processes of collection and processing to obtain a result Neverth^ss ncAwdy shouki rdy on a 
sii^e measure to make a diagnosis (your temperature is up, titerefore you have Inibonic plague); 
rather doctors and users of performance indkrators need to use a naige of , observations - and 
expertence - to draw suitable conduskms. A single indicator is a veiy poOT diagnostic tool 
However the indicators taken together do need to be comprehensive enough and carefully 
designed in order to f^vkle insights into what is really happening to aUow us to draw 
condusions and make decisions. 

There has also been a slow but steady changp kt die role of the g^ieral practitioner. This has been 
accompanied by an increasing awareness of die factors which affect our state of health. Thus 
general practittoners are movii^ more frtMn a sitaiation of treatii^ symptoms as they are presented 
(you come to me with a problem and ITl fbc it) to a system whteh promotes greater individual and 
community 'wellness^. This is a recogniti<m Uiat one's state of health is, in buge part, a reflectton 
of that indivkiual's lifestyle which, in him, can be controlled to a veiy great extent by the 
individual themselves. Certainly illness still has to be treated, but a lot can be prevented, and the 
quality of life improved by the individual realising their health is their own responsibility. 
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Agaiiv so it is with perfbnnanoe indicatcns. Just as going to tine doctor to be 'cured' may be a 
denial of one's contributicm to getting sick in the first pla«e, so it is if one denies one's 
re^nsibOi^ to manage and be aocountalde for tite educatkntal prooKses uncter one's control. 
Thus, p^haps, managenwnt and accountability are not all that inooo^tible. They are however, if: 

■ the individual or ori^i^tion is made aoowntable for adiievii^ ouhxnras, but is not 
actually g^ven the authority, vKponsilrility and sneans for ediievii^ flwm (i.e. the t^l 
omtrol Ites dsewha«); 

■ the outcomes agaii^ «4ik}t performance is measured are unreasonat^ and not adUevable; 
that is, the outcomes agreed to actually do not relate to something the oiganisatton or 
indi^ual can actually do; and 

■ the measures or diagnostic features chosen to judge the levd of performance are 
inadequate for the task. They have to be reasonable and valid measures of dw featiire and, 
hems;, have to be carefully deigned too! 

Likewise, improvement in one's state of heilth (and general quality of life) can be adiieved 
through attention to key factcfs responsibte for monitoring and in^ioving one's physical condition 
and lifestyle. Attentim to dte equivalent features win le^ to improvements in quality in 
vocational education and training. All (rf d^ oomes back to a sense of profossfonalisnv and a 
recognition that all individuals wd oiganisationa] systems within die vocational education 
'oiganism' contrilnito to the maintenance and improvement of its health and wdl-bdi%. And - of 
course - the prevailing 'culture' of a society and key individuals within it are responsible for 
creation of the orjganisattonal systems. 

The analogy starts to break down at this pohit, ho%vever, because most oiganisms work to preserve 
a homeostatic state - which maintains qr^ems in b^ce within an acoeptoble liormar rai^. The 
prtAAem for vocational education, and haice for setting perfonnance indicators, is answering the 
question - What is normal? While colleges, departments and other (»ganisational units attempt to 
maintain a state of balance, there needs to be allowance for diversity. The range ctf ways of 
deliverii^ programs and courses, the balance between intrinsically lower and higher cost courses, 
the problem of providing courses in rural areas or by distance education or open teaming makes 
thesettii^of bench-marks very hard indeed (although tai^ can be set more readily). Itmakes 
comparison difficult too - unless it is oon^MuriscHi over time wifiUn an oiganisatton or 
oiganisational unit, and in rdation to its goals and the achievement of mutually agreed targets. 

However the medical analogy may come to our aid again, because aldwugh systems tend to be 
homeostatic they operate within a range dictated by drcum^anoes and Ufestjde (e.g. dtey are the 
intrinsic factors which contribute to the 'actual' cholesterol level (and which may be beyond one's 
control) balanced s^ainst Hhose demoits of one's behavtour wlddt can actually help to control tlw 
levd). However fust as a society and the tatdividuals in it are respori^te fior the creation of 
organisational systems in education so it is in medicine. Payer (1990) points out in her comparison 
of the medical culture in four devetoped western countries C^ranoe, the UK, the US and (Uten) 
West Germany), different countries set different bench-marks for particular diagrtostic indicators 
which dictate whether or not an individual patient will be likely to obtain hirther treatm^t For 
exan^Ie: 

A few weeks later I came upon an article dealing with the new dn^ treatments aurod at 
lowering cholesterol. In the article, a Canadian doctor alleged duit while in England 
doctors start treating patients for hl^ dtole^totd (mly wton a patient's blood dtolaterol 
level reaches 300 miUigrams per dedlitre of cholesterol, in the United States treatment may 
begin at 225 millierams, and some doctors are suggesting ev&i lower levels. 
^ (Payer 1990, p.l53) 

In medicine, like education, bench-marks are flexible because it involves the decisions of 
individuals and the influence of culture. Therefore organisational form and culture needs to be 




borne in mind both in setting bench-mazks and making oompaiative judgements. Comparative 
judgem^ts may tiierefore be poorly infonned because tiiey are made on the basis of comparing 
cnganisational fractures and cultural framewoiks which are not Ute same. 

In relation to peifomiance indiatofs, the balance aj^^eais to be ^lifting from a predcnninant 
concern for accountability on tiie one hand, to one <rf quality inqnpvanent on other. 
Performance inditators have a dear raHe in mana^nrient and in i mprovem e n t Iheyare, if 
|»^^erly omodved, a way of diag^Kwdng illnesses before they become tenninal and of providing 
guMes for flie maintenance and improvement of oiganisatkmal health and wdl-being. 
Aooounta^ty (while it has its prooesss) is more of a state ndnd - althoi^ it is important for 
those both inside and outside TAFE. If tiie debate over perfonnance indicatois hdps to |»t»note a 
better state of mind which recognises and respcmds to accountability requirements so much the 
better. Ultimatdy otiier weapcms sl^uld be found for forming judgemotts and oifordng findings 
because respect for the perfbnnanoe indicator law' (or lore?) wiD only come if perio^^ 
indicators are perceived as rrapecting and protecting evaybody's rights. 

A plethora of issues were raised in the previous diapters and in an earlier Centre publication 
(Guthrie 1988). This section aims to ccmsolidate and discuss a number of the key ones amoi^ 
these. Several issues have already been considered in the medical analogy |ust discussed Others 
indude: 

m the quality of data and other information collected; 

■ the focus for the development and use of indicators; 

■ the rote of perfonnance indicators in social Justin. 
Data quality 

Recent experiences with databases at both the national and college levels lead me to 
express some concern about die quality and appropriateness of tihe data they contain. In 
this I am not alone. Many of those working within TAFE systems and colleges would 
share this concern. 

In a nutshell sonte of the issues and problems are these: 

■ the data collected from various sources are variable in terms of titeir quality, 
completeness and cffiisisttrKj^ 

■ those oi^anisations actually collecting the data pereeive veiy littte benefit from the 
odlection process for themidves; 

■ the various databases have not been designed in an integrated way so that the 
production of petforonanoe indicators and the use ctf indi^tors and oth^ data in 
managon^t becomes tedious; 

■ databases are often not constructed to answer rdatively simple management 
questions or provide really appropriate information or data. It may be difficult to 
combii^ data from a range of databases. In otNr cases tiie quality of the que^ons 
asked of the existing data is poor - for exampte askii^ how many students there 
are in TAFE is difforent from asking how many enrolments there are. A number 
naively believe these questions are the same; and 

■ the data gathered and the performance indicators derived from them may 
increasingly not reflect the reality of what is happening. 
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These issues and problems are all wdl recognised and attention is bdng paid to them at 
the national, State/Territory and college levels. Neverthetess, ti^se issues mainly rdale to 
the collection and checking processes used and the technical aspects of both the data and 
^databases. Ilie quaUty of data also has anotfier dimoidon, that of its a|^m>i»riataM^ 

This abject gets back to ^ to(A qttestb>ns in data gathering, Inchidii^ Why are we doing 
it? How will we use t^« data? Recently s^nificant efiim has been dedicited into 
specifying and defining the data collected nat kmally a nd at State/Teiritoiy terel. The 
question to be foced luwever (and one v^iich C0TT5 wiU soon be Invest^ttng) is 'Are 
the data we are collecting the data we really need? 

This gets bade to the informational needs of thc^e ^tfiering the data (on tite one hand), 
and those who %vill use it at otle- levds within and cmtside TAPE. It also relates to the 
appropriateness of data coUeded in tenns of what TAPE and (potoitially) other training 
providers are actually doing. For ocant]^, as tte ways of delivering training diversify, as 
competency-based training is introduced and as more open learning and s^-paced, 
workplace-based or on-the^ob training occurs, the nature of TAPE tead»ers' work will 
change. At present the concept of te»ha: contact hours is enshrined both in industrial 
awards and in the agreed key national performance indicators. In Ute future, less ooncmt 
may be paid to process-oriented indicators and more concern to indicators which reflect 
the outcomes of training, giving TAPE systems, colleges and teadias an increased freedom 
to determine how titese outcomes will be achieved. Therefwe tite time is ripe to examine 
the quality of data collected boUi in terms of what is ooltected and how it is collected and 
how it is to be used. Neverthdess this has to occur within a framewoik of the data needs 
of tiiose botti within and outside the vocational education and training system, and at a 
variety of levels. 

It win also be In^xnlant to ens»n« Aat data from all registeied provides of training are 
captured to ensure that the full extent of training (by whatever means and wherever it 
oocuis) can be aocuratdy collected and analysed. However, at present only a part (albeit 
the sigitificant part) is csq>tiued and consolidated. 

The focus of devstopment 

It has become dear both from the work undertaken at COfTTS and from the central role the 
TAPE National Centre now plays in the collection, proces^ and reporting of the TAPE 
statistical collection, that the focus for devdoping a viable system of perfMinance has to 
shift At present, the devdoiment of the key national perfbnnanoe indkators is fw 
cons This is aj^nopriatc, since to (fevetop a nationally consistent set of indicators 
requires a national committee. 

Nevertiteless the data on whidi the indicators are based are obtained by each 
State/TeiTitofy TAPE agency from data supplted by tiwir re^KCtive colleges and other 
institutions. Some data oom^ to the TAPE National Centre from rdativdy small 
independent colleges. Oiher training ocnming under a range of natkmal initiative (e.g. 
Jobtrain,Skill8hare, and some further educalicm) are not part of this system. TherefOTe,in 
the end, the quality of both tiie national data and tite national peifonnance indicators are 
very much dependent on the quality of the data supplied by colleges and other providers 
of trainii^ and forther education. 

Of th(^ colleges contacted in the course of this and other ]m>jects (and it must be stressed 
tlttse are r^tivdy few) the messages that came acn^ are: 

■ more use could be made of much of the potential management information and 
oth^ data available within a college; 
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• managanent information systems (even ^dent records data) tend not to be well 
integrated; 

■ in collection, vaiying interpretations are placed on definitim of key data 
denents (e.g. student enndment, attrition and retoition) which raise ocmoenis 
about the o)n^»Fability of coU^ data internally Oet aloiw wittt other colleges, 
TAPE systems and other provides); and 

■ the process of supplying data to central offices is seen as a burden rather than a 
^iiv-off of their own processes. 

This reflects the need to focus, in the first instance, on the management information and 
data needs of colleges and other smaller providers so that these are met Nevertheless the 
data needs of central office and other bodies within and outside the vocational education 
and training system need to be considered In the development of college-based systems. 

At in^^ent it app^s fliM ocdl^s and ti^ oiganisatkmal units feel little ownership of the 
data that are routinely collected. A number of States are examining the role of the college 
in the course of redevdoping their data modds. However some States still focus heavily 
on a centrally co-ordinated process of data gather^ 

Other States, for examjde South Australia, have colleges involved (in many cases heavily) 
in the devdopment of their own peiicmnaiioe indicators and data systems. This has led to 
colleges becomii^ aware of (tefidencies in thdr data collection procedures. These can then 
be addressed. Other providers of training have yet to be consulted about tiieir needs> or 
even invited to contribute ^tistical data. 

Thaie are, however, other reasons for proposing this dmnge in direction of data collection. 
The first reason is data ccmpleteness. In addition, trends overseas (for exampte those 
recently introduced in the UK) propose the removal of ccmtnd of TAFE-Uke colleges from 
local BuUiorities. In Australia this would be the equivalent of transferring TAPE colleges 
fnnn thdr presoit State/Territ^ control to federd control (particularly in relaticm to 
funding), in much the same %vay as in higher education. Indeed momentum for the 
increasing 'nationalisation' of the vocational education and training system is gathering on 
a range of fronts, indiuiii^ curriculum and Iraming restwroe devdo^nent, standards 
settii^ course accreditation, ete. If, therefore, the ccHitrd of TAPE colleges and the Uke 
becomes more kxalised this will enhance the need for hi^ quality mani^ement 
information at that level. (States and Territories are also inaeasin^y decentralisiiig 
management functioits.) Furtto*, pr^^ire to redepfoy funds aivay fhnn colleges not seen 
to be performing to Aose who do perform (by whatever criteria performance is measured) 
is also a trend in the UK. 

If learners are given some form of voucher (although it may be called by some other name 
in Australia) as they are in ihe UK to ^pend on post-compulsory education, th(^ 0)lleges 
who provide courses and ofho' services that are needed and are af^>ropriate (that is, they 
attract customers who want to buy the SCTvice) will prosper. Others will not This very 
pragmatic approach suggests need to focus on issues related to strategic planning, 
management and indicators of performance to ensure that colleges car. poK^ticm themselves 
both to adapt to emerging needs and priorities and, ultimately, to prnvide appropriate 
services. Nevertl^ess, this may lead to some of the moral dimension of the vocational 
education aiul training systan's work bdng overlooked. 

The second i^ue relates to quality assuraiKe systems at the college levd. A recent 
publi^tion of the Department of Education and Sdence, the Department of Employment 
and the Welsh Office in ttte UK {Education and training for the 21st centunp the challenge for 
coB^) addressed the Issues of quality assurance pointing out that great importance %va$ 
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attached by government to systems at the college level for monitoring and assessing 
quality (HMSO 1990). 

Thb report goes on to distinguish between tlw quality assurance rotes of. 

■ odleges; 

■ external asKssors; and 

■ examining and validating bodies (the closest paralld in Australia bdng those 
bodies responsible for the accreditation of fonnal courses). 

In the proposed system colleges have primary responsibility for quality control. As the 
report states: 

Most [oolteges) already have mechanisms for assessing the quality of tilie education 
and training diey jKDvlde. FerftnmaiKS iralicators rdated to qu^ity are being 
devdoped as part of college management information systems. Collies need 
effiective systons to improve tfieir quality and contribute to their own efficiency 
and elfecthreness. It would be jnemature to advocate any <me framework, but a 
number of systems are being explored by cdleges including BS5750, Total Quality 
Management (T(^ and Strategic Quality Managonent (S^fl). C^^geswillbe 
expected to pmvide infonnation to the councils about the quality assu ince 
systen^ they have in place; > • • (aiul) 

external asscason are responsible for making an irulq)endenl judgement of the 
quality of teadiing and learning in colleges, and for offering advice. The two main 
sources of Uds ectomal a^esanent at ]»esait are Her Ms^g^s Ii^xctorate and 
the Lo^ Education Auttiority GJE^ advtos. 

(HMSO 1991, p38) 

In this sense the comments underline on the urgent need to develop more formalised 
approaches to strategic planning and total quality managCTi^t witiiin oxlleges. These will 
include use of such methods as quality drcles, pofodic evaluations and reviews and 
performance indicators. Chapter 4 dealt at some loigtfi with these issues. Queensland 
TAFE (see Chapter 3) has internal revtew processes in i^ace and a rai^ of colleges have 
or are making use of these. Their 1989/90 annual report points out that the Institutional 
Review and Dev^pmoit Branch: 

• facilitated seven colleges in completing their self-study and review processes; and 
that 

• three other colleges were neaiing the end of their investigations, while a further 
three had begun the process. 

The annual report suggested that there were strong indications of satisfaction with the self- 
study and review process. It was also reported that several college had established 
permanent committees to support an ongoing review function within their resjjective 
colleges. 

In conclusion there is a dear need lo build on the drive to improve the quality of the data 
collected at their sotm»s throu^: 

• better ^tegic planning at college level; 
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■ compliance to a code of practice for evaluation and review p ro c ess es and the 
gatherii^ and use of management information and associated performance 
indicators. (CompUanoe witfi a variety of codes of practice can be performance 
indiratois in thonsdves.) 

■ the development and diseminaticm of a range of appropriate college-based 
management information systems; and 

■ the devdopment and dis^ninaticm oi a range of fnofe^ional ai^noaches to 
judging and improving the quality and appropriateness of the programs offered by 
colteges. 

While the latter two key issues have concentrated very much on the pragmatk dimensbn 
of performance indicates and the related managennent information systems and processes 
of which diey are part, it is an too easy to foiget the balancing moral dimen^n. Thiswill 
be ccsisidocd in the lolk>wing secticm. 



Indicators and sociai justice 

The j»;^matic discussion oS pofonnanoe indteittors is balanced by its moral dimoision 
(Guthrie 1988a). The key national performance indicators include only one soda! justice 
indicator, alttUHig^ a rai^ of TAFE agei^tes report that they suj^xirt a number of 
poformance indicators in this area, very often at the individual program level. These are 
usually directed at particular taiget groups (women. Aborigines, sii^gle parents^ migrants, 
the chronicaUy unempk>yed, people in nnal areas, etc.). As the Queensland lYeasuf/s 
puU'cation on developing per for it na nce indicators suggests indicators in this area should 
address some or all of tiie ftdlowing issues: 

■ equity - a fair distribution of resources awtm$ client groups; 
• eqi^ty - equal dvil, legal and iraiustrial rights for all; 

■ access - equitable access to services; and 

■ participation - the opportunity for individuals to participate in the decisions which 
afifect their lives. 

(Queensland Treasury 199(k:, p.lO) 

It follows irom die above that support needs to be provided to en^e individuals from all 
groups to meet the entrance criteria for courses and to succeed wifliln these courses (e.g. 
provision of study ddUs/Uteracy courses, etc) Tlito inchides chiMcare facilities. 

Neverthdess non-taigeted pro'^rams have a range of social justice impels as well, 
although the indicators of thefx success are often only ejqpressed in broad tenns (eg. an x% 
improvonent in partkipation i»y women in non-traditional programs). Only some measure 
of disaggregatton of the data woitki give some guidance about the extent to whidi 
particular couises and/or colleges had contributed to the achievonent of this target 

In addition, proWems remain in terais of trying to isolate the extent to wh^ participation 
groups are represented. Women, Abori^nal people and age are three factors whidi can be 
isolated rclativdy easily because they form part of the data collected nationally. Rural 
people and other groups can be detected by postcodes for home addresses. Hiese 
postcodes can also give some guide to sodo-economic status. Nevertheless other groups 
(nonenglish speaking buikground, single parents) will most likely be detected only if they 
are part of a specific targeted group within a progrant. For the most part Aese would be 
either specific programs (e.g. Jobtrain) or leameis in mainstream prog^wns wlme spedRc 
selection processes are aj^pSed. In some cases membership of particular targeted groups is 
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manifeted; in outers it may be disgui^d. Thesehne the levd potomanoe may be 
difficult to detennine with any accuracy. However, this raises the th«ny question of the 
in^^on of |Hrivacy throu^ asldng questimts on enrobnatt fonns and/or dsewhere which 
enable maid>»9hlp of a taa^ ffoap to be detOTnined - and wheti^ sudi data should be 
made available by colleges to central offices or at a national tevd. Alternatively such 
issues could be dealt with by periodic surveys using a nationally agreed methodolc^. 

To ensure that sodal justice is met, indicators mig|it address the following specific factors: 

■ gender inclusive curricula, t^dting methodologie' md matolals; 

■ freedom from sex-based harasanent; 

■ safety on can^Mis or wherever courses are provided (indude^ but not limited to 
OH&S, e.g. a well-lit, secure ranqws where students can saldy attend evening 
course^ using the Ittnary at n%ht, e^); 

■ recc^;nition of, and credit for, prior laming indudii^ overseas qualifications; and 

■ courses provided for under-represented groups articulating with mainstream 
programs. 

In short, ihae is a need to establi^ guideUiws m social justice, compliance witti which is 
another indicator of performance. Such indicators are additkmal to Htyose ccmcemed with 
measuring the extent to whidi proposed targets have been met or outo(»n» achieved. 

As the Queoisland Treasury (19^) point out: 

. . . social justice is an aspect of program ^iectivmess: social justice concepts have 
always been a principal motivating factor for gov^nment programs. 

Nevertheless insuf^oit thou^t has probaUy beei given to measuring the social justice 
a^MCts of the vocational education and tiainii^ system's performance - in particular the 
potential and often intangible social consequences of its various activities. These are, 
however, very difficult to report, much less quantify. They are also sulked to the same 
omoems as those of otha outcome- oriented peifoimanoe indicators and are t^ up witl. 
determining the proportion of credit (or blame) that the sjrstem can take (or incur) for 
improvements (downturns) in the sodal drcum^ances of those who use its sovices. 

Conclusions 

It seems appropriate to draw a number of oondu^ns based on tiiis survey of current practice in 
Australian TA^ and the review of the literature from both Australia and overseas, and both the 
school and higher education sectors. A number are contained in the concluding sections of the 
two previous duiptors (see pages 29 and 46). In this diapter ihe condu^ns are spUt into two 
types - key findings, aivl some directions for the future development and use of performance 
indicators. 

The key findings 

The key findings include: 

• there is a significant devdojnnait in inters in performance indicators for the 

purpose of local plarming, mani^ement, dedsion making and improvenrtent This 
coiiwides with a reawakening interest in the developn^nt and use of indicators 
both at ihe State/Teiritoiy aiul national levels; 




the quality of data gathered is poor. Infbnnatton is misdassified (e.g. field codes) 
and definitions are a^fdted inctmsistaitiy within and between reporting sources. 
Mcacover much of tte data t}»t are g a th e red remain tmdenitilised or even 
unanalysed (for example the fNiblished national data are not disaggre^ted by 
stream or by field of study). The (juality of question asked of tite datab^ has 
sometimes been poor too. As pointed out previously numbo* of students and 
number (tf enrolments are different concepts^ but are readily confused if the 
questiiHW or ttiose answering tiie questton are ui^ear about Hhs nature of the 
information they want The use of data over a time series is also jwDWematic as 
systems (tf collection and processing have either been changed or refined. This 
makes it difficult to determine wheUier a|^>arent trends are real or ortefocts of the 
changii^ datii gathering and mai^ulatkm process^ 

sectors of, and participants in the vocational education and training systems are 
nesteded in die national statistics While some data may be collected these data 
are not consolidated. Alternatively, data frrni a range of private and other 
provMers are not collected at an. As articulation, credit tran^ and recognition of 
pior teaming beonne more ^^fiont issues, it %vUl beomie increasingly 
important to reflect the ccnnplete contribution of a variety of providers to 
Australia's omall trainii^ ^rt; 

the need for a better integration of existing databases and more complete 
information about the training effort that TAPE and other providers of training. So 
far the ^fbrts of fhe private previdos of tiainii^ has recdved rdativdy little 
attention, but %vill need to do so as trainii^ options become more diversified and 
nwre complex; 

the debate continues over whether performance indicators are tools of 
accountal^ty ami control or are cme of a number of managonent instruments for 
Ote improvement of quality. At the local levd, where fnedominant use will be 
made of indicators, taprovement rather than control will be the major focus. As 
the indicators move hitter throu^ successive aggregations and management 
levels, they will increa^i^y become aocountabflity-locused; 

the nature of TAPE and the training environment is changing. With this, the 
information needed to manage and develop the vocational education and training 
system further will change. Skidi initiatives start at the tocal or ooUege levd, but 
as government policy initiatives (such as the Introduction of competency-based 
training and all it entails) beings to bite, so the demands for information will 
change and existing indicators will become increasingly iirelevant because they 
will be less able to caj^ure validly the 'essence' of ihe system's performance; 

a balance needs to be maintained in the use of paformance indicators to support 
TAPE'S pragmatic role in training and industrial refomv together with TAPE'S 
moral role to assist those in the community in general, but in the workforce in 
particular, to partidpatu in the changes occurring in Australian society; 

a range of indicators are needed to reflect properly the diversity of activity in the 
ai«as of further education and vocational training. Thus a series of questions may 
need to be asked in order to provide a dear picture of perfomiance to assist the 
manag»n^ and dedslon-makii^ processes. Indirators will be boUi quantitative 
aiui qualitative and may need considerable experience in their interpretation and 
use. TAPE and other bodies (pos^ly other providers as well) need to become 
more serious about tl^ use of management information tools. 



Future deveiopments 



litely future devdopirants in ttte use of perfonnance indlcaitors indude: 

■ the developmau of inani^rarant infonnation systons v ^c^RcaSfy de^giwd for use 
at the coU^ or loc^ level. There is a real need to una^take rattensive work on 
the dev^pment, implmentation and use of management infonnation systems and 
their related peifbnmnce indicators whidi are suitable for use at the college level 
or its equivaloit A refocussing at the cdtege tev^ more fncdse definitkms, inore 
staff dev^pmeni and other ^gnifkant efioits are needed to dramatic 
improvements in data quaUtx# usefulness^ and use. Moreover attaition will 
increasingly need to be g^ven to the validly of a nunto of the present national 
indicators as the m^ns of providing training change and dlvei^. AD of this will 
be an important part of the NATMIS5 proje^s woik In additiim a Itow to do it' 
manual on performance indicators for the college or local levd maybe a useful 
initiative;; 

■ increased attention this use of indicators for comparative purposes, inchiding 
interstate and international comparisons. A related devdopment may be their 
increased use in decisions about resource allocation, whether at the local, regional 
or State/Taritory levels; 

• a proper examination of tlK data needs of a variety of individuals and groups at a 
variety of levels is needed to develop a well integrated national data model, or a 
set of compatible data modds> which will cope with the dtanging infonoiation 
needs of thj»e within and watching tte Ausbalian ^^xational eduoition and 
training sjr^em; 

■ the issue of bench-marks aitd ihs vsAae of programs in relation to tteir context and 
role needs serious con^deration. Because we haw had relatively Uttle experience 
in the formal use of indicators and boich-marks whetih^ at tl« local, 
State/Territory or national levels we need to be relativdy flexible about bench- 
marks and targets to begin with. Expoience in tiieir use will help us both to more 
effectively set boKh-marks and targ^ and to determine the nonrnl "range' within 
which they should fall; 

■ survQT apjproadnes may be used more extrasivcly than they have in the past for 
determining satisfaction with service, destination and the participation of key 
interest groups in govonment policy. Increasii^ use may also be made of 
compliance audits, with th^ associated checklists and standards (which might be 
based on, or advised by, the rdevant British standard); and 

■ codes of practice ami system standards will need to be developed and agreed to in 
Older to guide tlw use of maiu^i^ information sptena and the conduct of 
evaluations, reviews and ^irveys to ensure that tl«se are conducted appropniately. 
In most cases these can be adapted from existing codes. There will need to be 
national agreemoits if data aiul performaiH^ indicators are to be comparaUe. 

The devetopment of indicators within a wider management and accountability context will 
continue. Nevertheless caution is needed while processes are developed and experience * 
gained. Mistah^ will be made, but it will be important both to share experiem^ and to 
learn from any mistakes rather than be draggi^ down by them through our own, and 
others', unrealistic expectations. 
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Appendix two: Indicators and collection 
processes 



GRADUATE DESTINATIONS 

This indicator is not collected 1^ 

Northern Territory 

Tasmania 

Victoria. 

Australian Capftal Tenttory 

Graduate destinations 

(graduate of A /Diploma, Adv. Cert, and some Certificate courses). 



Annual Surv^ by questionnaire. 
Careers Education. 

Published report The key to entfAayment. 



Collecting 
How? 
By Whom? 
Reporting 
How? 
By Whom? 

The information is reported externally by the ACT Institute of TAPE. 

New South Wales 

Graduate destinations 

(post-course oi^ymoit and furtiier tiainii^ outcomes for 6 Industry Training 
Divisions (TTDs) • collected for the fbllowii^ industries within ITDs Gnformation 
Technolo^, Tourism and Mortality, Ffaianoe and Inveshnent, Aviation, Urban 
Horticulture, Fashion)). 



ftirvQ^ of recent graduates in selected courses for 6 ITDs. 
6 ITDs. 



How? 
By Wlwm? 
Reporting 

How: Selected tables. 

By Whom? 6 ITDs, 

The infonmation is reported externally in the Annual Report. 

Queensland 

Collecting 
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How? 



Questionnaire. 



By Whom? State Office. 
Reporting 

How? Report. 

By Whom? Flanning and Perfonnance Analysiii Branch. 

This indicaUxr is for internal publication and distribution. It is used externally vthexe appropriate. 

Wmiwii AumDs 

WA DTAFE has conducted three annual graduate destination siuveys. Although the information 
has cm^eiaUe utility for performance measuronent it has not beo\ formally constituted as a 
petfonnance indicator. 

The first two surveys were conducted by the TAPE Counseling Service and reported in published 
repwts. They wse piindpally orioUed towards providii^ information for counsdlii^ prospective 
students. The tiUrd and most recent survey was conducted by the Curriculum Branch and was 
inteivied to have a more general utility. A draft report was prepared but ivas not widely 
distributed folfowing concern about the diffuse nature of the objectives served by the survey. 

It is plaraied tiiat information collected tiuou^ ^dent surv^ will be constituted as performance 
iiwlicator^ as part of the TAPE Learning Managemott Sy^em (LMS) currendy vani&r developn^t. 
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EXAMINATION PASS RATE 



This indicator is not collected 1^ 
Northern Tenitoiy 

Victoria (see note on Bgure 2.1 in the main report). 



Note <hat the October 1988 meeting of the Perfonnance Indicators Plannii^ Group 
reoommended that the examination pass rate be renamed Uie Subject Pass Rate (i«. the 
number of ^wtaits who sucoesshilly met die requir e moits of the subject divided by 
tiie number of students who enrolled in the subject) 



Auttrellaii Capital Tonftory 

CoUectitOL 



How? 
By Whom? 
Reporting 
How? 



Extraction from comput^lsed student cdlection (OLSAS). 
Student Administration. 



Acadonic year repenting, ijs. by March of the following year. Tl^ imlicator 
measures data by school course sut^ect gender and award. 



By Wh' #n? Education Program, ACT Institute of TAPE. 
The information is for internal use only. 
New South Wales 
Collectine 

G>nected for all major awmd courses through central computo* ^^stem. 
Student Administration. 



How? 
By Whom? 

How? 
By Whom? 



By subject, course, ooU^ ami ITD (bidustiy Trainii^ Division). 
Statistics Division. 



The information is report^ externally in tiie annual report as an average for selected courses in 6 
ITDs (Infbmsatfon Tedmolt^, Tourism & Hoqritality, HnaiKe & Invi^tment Aviation, Urban 
Horticulture and Fashfon). Internally the informatfon is available on request 

Quoenaland 

Collecting 

How? Examination data base. 

By Whom? Slate Office. 
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How? 
By Whom? 



Spreadsheet. 
State C^oe> 



The infoimation is not yet publisl^. 1990 was a Isench-maik' year. The information win be 
inoorpoiated in a puldication on CoIZege openOiond ^ ec Uvaie ss. 

TtsmanSa 



How: 



These data are entered at the college level for eadi sub|ect and stored on a 
centralised database. Calculations are made partly dectronically with a manual 
ounponont 



By Whom? Statistics Officer, 

How? Calculation is a foirly time oon«mnii^ pnooess and so r^rts are only produced on 

a linUted basis and only spec^ courses are tai)geted> OnnpaiabUity of sul^ects 
and courses is complicated by tite fact that differing sets of awards are used, some 
sul^ects use alphabetic awards others use numeric, while some subjects do not 
have any award attached. This system awnplicates electronic retrieval and makes 
manual manipulation of data awkward. It can be appreciated that tinis ff^stem 
makes (mdsion aloK^t bnpo^Ie with only rough 'gues^imates' being pos^e. 
Ctesnly if it was necessary to report on pass rates on a lai^g^ scale a ui^rm award 
^stem would have to be adopted. 

By Whom? Statistics Officer. 

These statistics are not published and are used on a limited basis internally by departmental 
WMiwn Australia 

This is a formal performance indicator coltected via die TAPE Shident Data System and reported in 
the Department's annual report in oomfdi^noe with die requirements governing performance 
indicators under the Financial AdminiMtion and Audit Act (FAAA). Taigets for this indicator 
have previously been set in college resource agreements, although the audit of the 1989 agreements 
cast doubt over tiie value of sudi an indicator givoi the manner in which it was set and reported. 




SAT EXAMINATION RATE 



This indicator is not collected by. 

Northern Territory. 



Note that it was reoomnnended at the October 1988 meeting of the FeifoixnaiKe 
Indicators Fkumtng Group ttiat the sat examination rate be renamed Subject 
Completion Rate, i.e. the number of ^dents who completed the assessable 
requirements of tite subject, divided by the number of shidents who enrolled in 
tite sul^ect. 



AustiaUan Capital Tenltory 

CoflecUng 



How? 
By Whom? 
Reporting 
How? 



Extraction from computerised student collection (OLSAS). 
Student Administration. 



Academic year reporting, i«. by March of itvs following year. Tlw indicator 
meaaires data by school, course subject, gaWler and award. 



By Whom? Education Program, ACT Institute of TAPE 
The information is for internal use only. 
New South Wales 
qpHe^g 

Collected for all ma^r award courses through central computer system. 
Student Administration. 



How? 
By Whom? 
Report ing 
How? 
By Whom? 



By subject, course, college and ITD (Industry Training Division). 
Statistics Divi^on. 
The information is available internally on request. 

Queensland 

Collecting 

How? Examination database. 

By Y/homl State Office. 
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Reporting 

How? Spreadsheet 
By whom? State Office. 

The infonnation is not yet {MiUished. 1990 was a l)ench-inark' year. The inftmnatidn will be 
iiKOiporated in a publication on Colt^ (^tenOkmai ef^dwetiess. 

Tasmania 

How? Calculated electronically for each subject Results, if given, are compared to 

enrolments to give an attrition measure. 

By Whom? Statistics C^cer. 

Reporting 

How? Calculated electronically on a once a year basis. Calculations give a percentage of 

those students who achieve a result in the subject Calculations are made for 
subjects and courses across colleges. 

By Whom? Statistics Officer. 

These statistics are not puUi^ied but are used internally departmental officers. 
Weasom AuatraUa 

See ttie relevant entry for examination pass rate. 




ANNUAL GRADUATE NUMBERS 



This indicator is not coDected by: 
Northern Territory. 

AuMtbin Capital Territory 

C9^ectinR 



How? 

By Whom? 
Reporting 
How? 
By Whom? 



Extraction from computerised student collection (OLSAS). It is collected for all 
formal awards as put of the annual graduation process. 

Student Administration. 



Graduate handbook. 
ACT Institute of TAPE. 
The information is published for externa! consumption. 
Now South Wales 

How? Collected for ma|or award courses throu^ central computer system. 

By Whom? Statistics Division. 

TMs information is reportai externally in the annual rqsort, nuntos by ITD only. Intranally, the 
information is available on request 

Qiioonsland 

Collecting 
How? 
By Whom? 
Reporting 
How? 
By Whom? 



Data sheets. 
Colleges. 

Computer printouts. 
State Office. 



The hiformation has not yet been published. 1990 was a ttench-mark' year. It is to be 
incorporated in a publication on CoUege operational effectitxnas. 
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Colleges manually work out which studnUs qualify for an awanl. They collate 
tl^ data and forwaid them to a centialised exaxninatlons section. At tiiis point 
data are entered for iSxose sHidonis who qucdify. This infomiation is tied to course 
information ami studoit data in the laiger database. 

CoHeded by cdleges and entered onto computer systan by operatois in a 
centtalised examinations section. 



Reporting is undertaken electronically using a query language. Data are 
downloaded onto a PC and manipulated. Comparisons can be made from 1988 
onwards for each course. 

Statistics Officer. 

This information is not published and is only produced for targeted counses on a limited basis. It 
would be possible fo run off graduate numbers for each course on a yearly basis. This information 
was required by the Commonweal^ as part of its statistics collection for the 1^ collection. The 
phrasing of that data requiremoit (element 150) was as follows The mmber of awards gmtel during 
the reference year hi respect of the course. Curraitly the data are only produced for staff members 
who require the information for iiuiividual res^trch projects. 

Vletorla 
Collecting 

How? Annual statistics for previous year. 

By Whom? Statistics Section (whidi is part of the College Budgets and Performance 

Management Section (CBPMS) located in the Resources Management Division). 

Reporting 

How? Data tape to DEET 

It was noted that this infoimation was not published. Indeed it was seen as usdess and dangerous 
informal m without substantial qualifications and notes. 

Western Au^ralla 
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Tasmania 
Collecting 
How? 



By Whom? 

Reporting 
How? 

By Whom? 



Annual graduate niunbers have been coUected at least for the past decade and have been used 
informally for performance measurement alttough not formally oonstihited as a perfonnaiKe 
indicator. 

Previously manually compiled this statistic is now compiled on the Student Data System. It is 
collected by the TAPE Examinations Branch. To date it has only been reported internally however 
it may now be featiucd in the Department's annual report. 

It is planned that course graduation rates will be a key performance indicator for DTAFE in the 
TAPE Learning Management System (LMS). 




Q2 Pwtoi r a no Memn nmm 73 



STUDENT CONTACT HOURS/TEACHING HOUR 

This indicator was collected by all States/Tenitoiies (see Cahpter 3 of this :>^p-A regardii^ South 
Australia). 

Australian Capital Toriltory 

CoHecting 



How? 

By Whom? 
Reporting 
How? 
By Whom? 



OLSAS (computerised student collection) for ASC34 and Teadiing hours from 
computerised full-time and part-time teacher duty hours system 

Education program. 



Managonent reports. 
ACT Institute of TAPE. 
The indicator is for intoiud constunption. 
Now South Wales 
Collecting 

Extracted from central compute systrn-i. 
Statistics Division. 



How? 

By Whom? 
Reporting 
How? 
'\y Whom? 



By Industiy Training Division (ITD) and State total. 
Statistics Division. 



Tlie information is published in DEFTs Setetcf TAPE Statistics. Available to NSW TAPE 
management on request 

Noittism Terr^ory 

CoUecthig 
How? 
By Whom? 
yepprting 
How? 



Enrolment and course forms con^^leted at college and /or regional level. 
ColSeges/regions. 



Reports to colleges ami re^ons and to central management. 



By Whom? Office of Tertiary Education. 

The information wUI be used internally until its reliability is established. 
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Quoensland 



I 



ierIc 



Collecting 

How? 

By Whom? 

Reporting 

How? 

By Whom? 



Centralised enrolment database and centralised teacher duty hours database. 
State Office. 

Spreadsheet 
State Office. 



The indicator is for internal publication and distribution. It is published externally where 
i^jpropriate. 

Taanania 



Collecting 
How? 



Contact hours for each course are stored electronically on the centralised database. 
This data elonent has to be kept as part of the Conunonwealth-State Resource 
Agreement. 

Course contact hours are calculated from set contact hour figures for each subject. 
These set subject figures are entared onto the database by offioo^ at a central office 
when ihe subject is created. Subject hours are multiplied by enroln^te for each 
course to give a course figure. 

Colleges return to central office a staff return once a year at the beginning of each 
year for the preceding school year. This r?tum shows the number of teaching 
tours for each college. As information is only avaiiaMe at the college level and 
not at ihe 

course level it can be seen titat Uus imiicator under present circumstances is of 
limited use. 



6y Whom? Co-ordinated by Statistics Officer. 



Rqoorting 
How? 



Reporting (at the college level) has been on a very limited basis where figures are 
odrulated manually using data from staff returns aiKi totals of contact hours for 
the colleges. 



By Whom? Statistics Officer. 

Figures are not published uid this indicator has only been used a number of times in the last few 
years. As noted diis information is only able to be produced on a yearly basis and only for the 
preoediii^ year. 

V(Ctc»l8 



Collecting 
How? 



Financial statistics collection. 
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By Whom? 



Statistics Section. 



Repoitinp 



How? 



Computer summary by ooUege; available also down to oourse levd. 



By Whom? College Budgets and Performara^ Monitoring Section (CBPMS) in the Resources 
><fana^ment Di\'{sion. 

The indicator is published internally but selected data are made available tor college managen^t 
and budget discussion purposes. Acts as surrogate average class size indicator - very useful for 
comparative purposes. 

Western Australia 

These are currently in use and will continue to be used as formal performance imiicators. Ihey 
axe comj^ed on the Student Data System by tite computing section and repented in relation to 
College Resource Agreonents (int^nal distribution only - although {banned to indude in the 
DTAFE Annual Rei»rt). 



^ ERIC 



76 Pwtonwm MtaMMB miatM] 




I 

I 



AVERAGE TEACHING HOURS/FULL-TIME TEACHER 



I 



lERiC 



Australian Capital Terrftoiy 

Cc 



How? 
By Whom? 
Reporting 
How? 
By Whom? 



Con^toised full-time teadter duty )K)urs system. 
Education program. 



Manag^nent rep(»1$. 
ACT Institute of TAFE. 
The imUcator Is for internal consumption. 
New South Wales 
CoMecting 

One week cenms of teaching prc^ms. 
Statistics Division. 



How? 
By Whom? 

How? 
Bv Whom? 



By college, ITD, and State total. 
Statistics Divi^on. 

The information is not published. It is avaUable to NSW TAFE management on request. 
Northom Torrltoiy 

Q>y?cti'>8 

Enrolment arai course forms completed at college and/or regional level. 
Colleges/regions. 



How? 
By Whom? 
Reporting 
How? 
By Whom? 



Reports to colleges and regions and to central management. 
Office of Tertiary Education. 
The information will be used internally until its reliability is established. 
Qusensland 
Collecting 

How? Centralised teacher duty hours database. 

By Whom? State Office. 
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Reporting 



How? 
By Whom? 



Spreadsheet. 
State Office. 



The indioitor is for internal publication and distribution. It is published externally where 
apprt^riate. 

Tmiania 



Collecting 
How? 



This indicator would have only a limited use at a central office le^; l^wever it is 
reasonable to expect that it is used within a>Ileges. Qven the annual ^aff return it 
would be pc^sible to make some calcidation for each colle^ and for the whole 
division. The annual s*aff return iS required by the Commonwealth as part of tine 
resource agreem^t. 



By Whom? Statistics Officer. 



Reporting 
How? 
By Whom? 



Very limited basis at central office level but posably greater use at college level. 
College staff. 
Not published aiKi used only internally. 
VIctolia 

Victoria does not presently collect information for this indicator. However it will be collected in 
future by the College Budgets and Perfonnance Monitoring Section (CBPMS) in the Resources 
Management Division. At present it is reported on an ad hoc basis to tlw CBPMS by colleges. In 
the future it will be reported by unit record. 

Western Australia 

Data on teachUtg hours per full-time tead^ are used in ccdleges for a variety of management 
ptuposes but IS not constituted as a formal perfonnance ii^icator. 



ERIC — 



I 



RECURRENT COST/STUDENT CONTACT HOUR/COURSE 
Note: A number of States cannot disaggregate these data to the course level 
Auitiallan Capital Terrttoiy 

Cpl^ectinfi 

How? Institute/scttfX)! budg^ papers and OLSAS (computerised student collection}. 

By Whom? FinaiKial management 

Rgportinfi 

How? Management reports. 

By Whom? ACT Institute of TAPE. 
The iivlicator is for internal consumption. 
Now South Wales 

Note that for this indicator data are not availaUe at course levd. However information is collected 
from the NSW corporate indicators on the Cost/Shident Contact Hour/SchooL This is based on 
salaries and consumable expenses. The data are collected by Finance from central financial records 
and r^rted by Sdwol in the Annual Report. In addition information has also been gathered on 
the total oost/student contact hour on a college ba^ for two colleges (North Sydney and Grafton). 
Th^x data were gathered by Finance and manually extracted from college records. They were 
reported bodi externally in the Annual Report and internally to respective college managements. 

Noithem Terrttoiy 

Collecting 

How? Enrolment and course forms completed at college and/or regional level. 

By Whom? Colleges/regions, 

Reporting 

How? Reports to colleges and regions and to central management. 

By Whom? Office of Tertiary Education. 

The information will be used internally until its reliability 
is establisl^. 

Queensland 

Collecting 

How? Centralised student enrolment database. MSA Financial accounting database. 

By Whom? Slate Office. 
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Reporting 



How? 
By Whom? 



Spre£uis!raet 
State Office. 



The indicator is lor internal publication and distribution. It is publisl^ externally where 
ai^yropriate. 

Tasmania 



Collecthig 
How? 

By Whon\7 
Retxwtinp 
How? 

By Whom? 



This indicator is used by program managers. Contact hours per course are held on 
a centralised database. Program managers make their own calculations based on 
enrolment data. 

Enroln^t data envied by colleges. 



Currently no set reporting format exi^ Contact !u)ur figures for each course can 
be run off at ti*e college level aiui at l^ad offi(%. 

Data processing officers at the respective locations run off monthly course figures- 
Program managers make Iheit own calculations. It should be noted that costing 
would be taig^ed at the college levd; it would be rate to find a widespread use of 
this indicator at the course level within colleges. At tte head office level recurrent 
cost per student contact tour pa course is being used, but only on a limited basis. 

Not published, these figures are used internally. Note that recently TAPE within Tasmania has 
merged %irith several other departments. As a «>nsequcnoe of this change the information needs of 
the department are bdng reassessed. It should be possible to electronically integrate the enrolment 
^tem with the perwnnd aivl finaiKial systems. When tiiis occurs (ami given planned changes to 
ihe Division's ^tem of budgeting) it is reasonable to expect that greater use will be made of this 
indicator. 

Victoria 

Collecting 

How? Rnandal statistics return based on: 

direct t^urhing costs x course, 
total costs X college, 

cost by activity, e.g. admin/library /student services etc, per student 
contact lK>ur by college. 



By Whom? 

Reporting 
How? 



The College Budgets ami Pesformance Monitoring Section (CBPMS) in the 
Resource Management Division. 



Computer printouts by course/course types - e.g. field of study. Also used to 
advise colleger on fee-for-service changes. 



By Whom? CBPMS, Manager Finance. 
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This is tte most widdy used ii^icator of college petformance. It is used internally for college 
«»nparison, budget discussions etc 

Western Austratia 

Cunent data collection ami methods of inputting cc^ts are not considered sufficiently reliable or 
accurate to be used for performaiKe measurement. 



Appendix three - Comments on specific 
Indicators by COTTS* 



1. 



Cliont assessment 



Three measures of this dimension of system effectiveness ate proposed: 



a. 



Student D^tination 



Student Satisfaction 



c. 



£0^10x0* Sati^ction 



These are to be collected using survey materials which have already been 
developed and triaUed in the ACT and a number of States, including New South 
Wales, South Australia, Queensland ami We^em Australia. Tine jnoposal is that 
industry grou(» be sampled in all States and Territories in a structured way «ich 
that all ii^u^es are asvered over a three to five year poiod. (There woidd be 
flexibility to include industries of national priority in sanies at short notice.) 



2. Education achievement 

2.1 a. Sat Examination Rate 

b. Subject Pass Rate 

Tlwse two imlicators relate to retention in, or comj^on of, subjects and other 
programs leading to de^red educational outran^ They have been used in a 
nuii^)a' of Statra/Territraies for two or three years to measure the number and 
propOTtion of students still on course at die end of their annual enrolled program 
and the number and proportion who complete the assessment requirranents 
satisfoctorily. National reporting will be at an appropriate level of aggregation 
such as Stream/Fidd of Study and may need to be rolled acn^ ^udy areas, 
depending on resources. 

c. Course/Program G>mpletion Numbers 

A tMroada* irdicator than Annual Graduate Numbers which is at course or 
program of shidy levd ratter than sub^ level. Because existing systems require 
students to enrol for an award or other course when they have no intuition of 
taking the whole course, diis imlicator will be of vahie as a time series of numbers 
not linked to comn^iKCXi^t numbers. 

22 Student Contact Hours/Direct Teaching Hours 

Univoisally used but in need of careful definition of populations and the teaching 
hours to be included. There is a particular difficulty with defining equivalents for 
distance education delimy and for g^ups such as Aboriginals in some States and 
theN.T. 



■ lliese coounenls ai« drawn from the CDTI5 piper prepared for VEETAC (COTTS I99I) 
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23 Average Teaching Hours/Full-time Teadwr 

Qear definiticm is required of die compoi^nts of &iis indicator; in particular the 
calculation of the nundwr of full-tinw teachera, or equivalent, may need to be 
related to prevailii^ award conditions for teadten in eadi ^te. Whetiier or not 
the demnninator of titte proposed indicator should be as stated or an EFT figure 
for an teadiers, and how ihs issue of duty tours and equivalents should be 
treated, also require» investigation. 

2.4 Cost/Shidoit Contact Hour 

Altiunigh sudi an indicator is in general use already, there are major difficulties in 
establishing and adtievii^ omsi^it finance data. The scope of the expenditure 
element will require car^ delineation, probably drawii^ on expol^ In the ABS 
and Commcmwealth Grants Commissioa A consistent and reU^le calculation of 
student contact hours is also problematic given TAFFs diversity of ddivery 
strategies and changes in the balance between them. Thus the devdopment of this 
indicator could well be the greatest technical problem facing COTTS. 

3. ACC088 

3.1 TAPE Participation Rates 

A linking of TAPE partkipation to population (initially could be student contact 
hours per 1000 population aged 15+) %vith possible jHOgrrasive exten^on to 
population sub-groups (e.g. women, ethnic ffmups, rural population). 

32 Unmet Demand 

The current DEFT collection is not reliable but merits high priority for 
improvement Work will be needed on medianisms for collecting this information 
%vithin a strict set of definitions and code of practice. A particular concern is 
duplicate applications by prospective students as TAPE systems generally do not 
have a central dearin^use qKration at dtho- tltt State/Tenitoiy or national 
level as does higher educatton. Records are hdd at provider level and 
incompletely, which malkss accurate recording difficuh to achi •ve. 

4. Commercial activity 

4.1 Industry-Funded Training/Total Recurrent 
To be developed. 

42 A ^alf development/industry experience indicator to be developed. 
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Appendix four: Key performance indicators 
(KPis) for the three areas of responsibility in 
the ACT Institute of TAPE 

1. Ec!iieatlon iteUvery KPis 

Effectiveness 

The ectmt to which: 

• course offvinf^ contribute to the economic devdofmient and social priorities of the 
ACT GovotUi^t; 

■ courses are accredited as meeting national edurational and industry standards; 
a annual graduate numbo^ increase; 

« student are satisfied with their educational e)^>erienoe; 
a participation of disadvantaged groups iiurreases; and 

• industry tiaining requiren^ts are met 
Efficiency 

The extent to which: 

a courses are fm>vided within budget; 

a student contact hours per teaching hour iinrrease; 

a class sizes permit ^fective learning; 

a consumables are used to brat advantage; 

a use of educational facilities is mardmised; and 

ft an appropriate balance between fulKtin^ and part-time teachii^ staff is 
maintalrad. 

2. Education services KPis 
The extent to which: 

a curriculum, teaching ^t^es and course profiles meet national standards; 

■ appropriate articulaticm ammgements Mrith other educational institutions and 
sectors pemiit ready transfer into and from TAPE rourses; 

a students, tl^ community ai^ industry are satisfied with TAPE services; 

a support services enhance student learning; 
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■ a^ess to, and paitidpation in, TAFE sovices (is) enhanced; 

■ educational services incrrase course completion rates and improve the graduate 
success rate. 

Coiporate services KPis 

Effov'niveness 

To die extent which: 

■ management information and advice is timely, relevant and ttxvrate; and 

■ provision of building and estate, pownal and other support services meet 
statutory requironent and user needs. 

Efficiency 

The eadtent to which: 

■ services provided by the Institute are adtieved within budget and meet 
performance standards of timeliness and quality. 
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